[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:21 dist-test-slave-8tdl krb5kdc[31927](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:21 dist-test-slave-8tdl krb5kdc[31927](info): set up 2 sockets
Aug 12 03:57:21 dist-test-slave-8tdl krb5kdc[31927](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 03:57:23.039981 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:57:23 dist-test-slave-8tdl krb5kdc[31927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:57:23.178094 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:38559
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:38559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:23.271677 31942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:23.271991 31942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:23.272047 31942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:23.273236 31942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:23.273305 31942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:23.273329 31942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:23.273352 31942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:23.273380 31942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:23.275290 31942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:38559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:38559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:23.275823 31942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:23.276150 31942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:23.284811 31951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:23.285544 31948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:23.285669 31947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:23.286729 31942 server_base.cc:1034] running on GCE node
I20260812 03:57:23.287158 31942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:23.287649 31942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:23.288803 31942 hybrid_clock.cc:648] HybridClock initialized: now 1786507043288765 us; error 63 us; skew 500 ppm
Aug 12 03:57:23 dist-test-slave-8tdl krb5kdc[31927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507043, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:23.291759 31942 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:57:23.292639 31942 webserver.cc:460] Webserver started at http://127.31.43.254:39743/ using document root <none> and password file <none>
I20260812 03:57:23.292970 31942 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:23.293027 31942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:23.293210 31942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:23.294448 31942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "c3874b93f2c6487d8ef640d641d4e6cb"
format_stamp: "Formatted at 2026-08-12 03:57:23 on dist-test-slave-8tdl"
server_key: "e259baef3f2c33be6d3229ce5711b8ef"
server_key_iv: "5901e6b25227f00c43ac2a2a3bc96844"
server_key_version: "encryptionkey@0"
I20260812 03:57:23.294934 31942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "c3874b93f2c6487d8ef640d641d4e6cb"
format_stamp: "Formatted at 2026-08-12 03:57:23 on dist-test-slave-8tdl"
server_key: "e259baef3f2c33be6d3229ce5711b8ef"
server_key_iv: "5901e6b25227f00c43ac2a2a3bc96844"
server_key_version: "encryptionkey@0"
I20260812 03:57:23.297116 31942 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:57:23.298910 31958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:23.299229 31942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260812 03:57:23.299348 31942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "c3874b93f2c6487d8ef640d641d4e6cb"
format_stamp: "Formatted at 2026-08-12 03:57:23 on dist-test-slave-8tdl"
server_key: "e259baef3f2c33be6d3229ce5711b8ef"
server_key_iv: "5901e6b25227f00c43ac2a2a3bc96844"
server_key_version: "encryptionkey@0"
I20260812 03:57:23.299439 31942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:23.310618 31942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:23.313100 31942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:23.313263 31942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:23.319540 31942 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:38559
I20260812 03:57:23.319608 32019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:38559 every 8 connection(s)
I20260812 03:57:23.320226 31942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:23.321805 32020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:23.325176 32020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb: Bootstrap starting.
I20260812 03:57:23.326138 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 31942
I20260812 03:57:23.326324 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:23.326494 32020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:23.326946 32020 log.cc:824] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:23.327289 31919 external_mini_cluster.cc:1428] Setting key c87390c515061994471803e47d3b92c5
I20260812 03:57:23.328253 32020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb: No bootstrap required, opened a new log
I20260812 03:57:23.330569 32020 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3874b93f2c6487d8ef640d641d4e6cb" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 38559 } }
I20260812 03:57:23.330843 32020 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:23.330940 32020 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3874b93f2c6487d8ef640d641d4e6cb, State: Initialized, Role: FOLLOWER
I20260812 03:57:23.331121 32020 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3874b93f2c6487d8ef640d641d4e6cb" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 38559 } }
I20260812 03:57:23.331257 32020 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:23.331301 32020 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:23.331352 32020 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:23.332816 32020 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3874b93f2c6487d8ef640d641d4e6cb" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 38559 } }
I20260812 03:57:23.333123 32020 leader_election.cc:302] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3874b93f2c6487d8ef640d641d4e6cb; no voters:
I20260812 03:57:23.333442 32020 leader_election.cc:288] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:23.333467 32027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:23.334002 32020 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:23.335544 32027 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [term 1 LEADER]: Becoming Leader. State: Replica: c3874b93f2c6487d8ef640d641d4e6cb, State: Running, Role: LEADER
I20260812 03:57:23.335700 32027 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3874b93f2c6487d8ef640d641d4e6cb" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 38559 } }
I20260812 03:57:23.336309 32042 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3874b93f2c6487d8ef640d641d4e6cb. Latest consensus state: current_term: 1 leader_uuid: "c3874b93f2c6487d8ef640d641d4e6cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3874b93f2c6487d8ef640d641d4e6cb" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 38559 } } }
I20260812 03:57:23.336432 32042 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:23.336712 32028 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3874b93f2c6487d8ef640d641d4e6cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3874b93f2c6487d8ef640d641d4e6cb" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 38559 } } }
I20260812 03:57:23.336773 32028 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb [sys.catalog]: This master's current role is: LEADER
W20260812 03:57:23.338085 32044 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:57:23.338158 32044 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:57:23.338248 32045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:23.338663 32045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:23.341560 32045 catalog_manager.cc:1347] Generated new cluster ID: 80e5566948734a78b330b885c6cf818f
I20260812 03:57:23.341641 32045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:57:23 dist-test-slave-8tdl krb5kdc[31927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:23.346450 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:23.329009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54976 (local address 127.31.43.254:38559)
0812 03:57:23.329338 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:23.329348 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:23.329536 (+ 188us) server_negotiation.cc:407] Connection header received
0812 03:57:23.331715 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:23.331722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:23.331825 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:23.331980 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:23.333972 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:23.335797 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:23.337490 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:23.337902 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:23.338024 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:23.342312 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:23.342317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:23.342317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:23.342330 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:23.344187 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:23.344639 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:23.344642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:23.344643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:23.344767 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:23.345058 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:23.345059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:23.345059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:23.345302 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:23.345546 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:23.345766 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 03:57:23.345945 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":109,"threads_started":1}
I20260812 03:57:23.374527 32045 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:23.375448 32045 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:23.379442 32045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3874b93f2c6487d8ef640d641d4e6cb: Generated new TSK 0
I20260812 03:57:23.379642 32045 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 03:57:23.418303 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:38559
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:23.555290 32052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:23.555526 32052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:23.555553 32052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:23.557646 32052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:23.557771 32052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:23.557842 32052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:23.561326 32052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:38559
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:23.561961 32052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:23.562325 32052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:23.572916 32058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:23.573272 32057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:23.573559 32061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:23.576140 32052 server_base.cc:1034] running on GCE node
I20260812 03:57:23.576521 32052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:23.576900 32052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:23.578063 32052 hybrid_clock.cc:648] HybridClock initialized: now 1786507043578045 us; error 29 us; skew 500 ppm
Aug 12 03:57:23 dist-test-slave-8tdl krb5kdc[31927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507043, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:23.582653 32052 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 03:57:23.583534 32052 webserver.cc:460] Webserver started at http://127.31.43.193:46081/ using document root <none> and password file <none>
I20260812 03:57:23.583874 32052 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:23.583961 32052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:23.584128 32052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:23.585803 32052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "a2917572d8894250a9ee69e2629108ed"
format_stamp: "Formatted at 2026-08-12 03:57:23 on dist-test-slave-8tdl"
server_key: "dd7e5b3041382afdf23f9ca2b0c2ea36"
server_key_iv: "b8892df04961bddb5ebd3fe86bc0b276"
server_key_version: "encryptionkey@0"
I20260812 03:57:23.586585 32052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "a2917572d8894250a9ee69e2629108ed"
format_stamp: "Formatted at 2026-08-12 03:57:23 on dist-test-slave-8tdl"
server_key: "dd7e5b3041382afdf23f9ca2b0c2ea36"
server_key_iv: "b8892df04961bddb5ebd3fe86bc0b276"
server_key_version: "encryptionkey@0"
I20260812 03:57:23.589078 32052 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:57:23.590231 32069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:23.590423 32052 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:23.590520 32052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "a2917572d8894250a9ee69e2629108ed"
format_stamp: "Formatted at 2026-08-12 03:57:23 on dist-test-slave-8tdl"
server_key: "dd7e5b3041382afdf23f9ca2b0c2ea36"
server_key_iv: "b8892df04961bddb5ebd3fe86bc0b276"
server_key_version: "encryptionkey@0"
I20260812 03:57:23.590591 32052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:23.604490 32052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:23.608523 32052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:23.608857 32052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:23.609366 32052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:23.610140 32052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:23.610271 32052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:23.610318 32052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:23.610337 32052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:23.629240 32052 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:34361
I20260812 03:57:23.629357 32201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:34361 every 8 connection(s)
I20260812 03:57:23.629772 32052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:57:23 dist-test-slave-8tdl krb5kdc[31927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507043, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:23.639313 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 32052
I20260812 03:57:23.639546 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:57:23.639704 31919 external_mini_cluster.cc:1428] Setting key f754711a6b1200d7d815b6889ae8c01c
I20260812 03:57:23.642472 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:23.630842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:38415 (local address 127.31.43.254:38559)
0812 03:57:23.630911 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:23.630913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:23.631334 (+ 421us) server_negotiation.cc:407] Connection header received
0812 03:57:23.632238 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:23.632240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:23.632259 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:23.632333 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:23.633728 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:23.634416 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:23.636055 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:23.636284 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:23.636375 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:23.639299 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:23.639301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:23.639301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:23.639310 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:23.640981 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:23.641414 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:23.641416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:23.641416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:23.641479 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:23.641767 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:23.641768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:23.641768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:23.641892 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:23.641962 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:23.642279 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 03:57:23.642375 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:57:23.642756 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:23.630855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:38559 (local address 127.31.43.193:38415)
0812 03:57:23.631189 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:23.631234 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:23.632113 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:23.632415 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:23.632420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:23.633263 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:23.633601 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:23.633608 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:23.634534 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:23.634535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:23.635949 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:23.635952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:23.636390 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:23.636391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:23.636604 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:23.637290 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:23.637299 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:23.639189 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:23.641073 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:23.641083 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:23.641086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:23.641335 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:23.641563 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:23.641564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:23.641565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:23.641699 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:23.641965 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:23.641974 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:23.642134 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 03:57:23.642243 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 03:57:23.642431 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":223,"threads_started":1}
I20260812 03:57:23.643441 32202 heartbeater.cc:347] Connected to a master server at 127.31.43.254:38559
I20260812 03:57:23.643544 32202 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:23.643709 32202 heartbeater.cc:511] Master 127.31.43.254:38559 requested a full tablet report, sending...
I20260812 03:57:23.644615 31975 ts_manager.cc:194] Registered new tserver with Master: a2917572d8894250a9ee69e2629108ed (127.31.43.193:34361)
I20260812 03:57:23.645565 31975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:38415
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 03:57:23.674319 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:38559
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:23.794857 32207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:23.795181 32207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:23.795215 32207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:23.796777 32207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:23.796833 32207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:23.796864 32207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:23.798731 32207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:38559
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:23.799047 32207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:23.799320 32207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:23.806226 32213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:23.806305 32216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:23.806321 32214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:23.806529 32207 server_base.cc:1034] running on GCE node
I20260812 03:57:23.806818 32207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:23.807101 32207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:23.808300 32207 hybrid_clock.cc:648] HybridClock initialized: now 1786507043808271 us; error 27 us; skew 500 ppm
Aug 12 03:57:23 dist-test-slave-8tdl krb5kdc[31927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507043, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:23.812011 32207 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 03:57:23.812573 32207 webserver.cc:460] Webserver started at http://127.31.43.194:45817/ using document root <none> and password file <none>
I20260812 03:57:23.812872 32207 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:23.812961 32207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:23.813143 32207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:23.814597 32207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "df523d794bbd45b3b03c4954f34ee0c4"
format_stamp: "Formatted at 2026-08-12 03:57:23 on dist-test-slave-8tdl"
server_key: "bb17d0c4444616f24ab628eb30b4c196"
server_key_iv: "d3a3359085e2b0f7a77c34b7c2eb86bb"
server_key_version: "encryptionkey@0"
I20260812 03:57:23.815191 32207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "df523d794bbd45b3b03c4954f34ee0c4"
format_stamp: "Formatted at 2026-08-12 03:57:23 on dist-test-slave-8tdl"
server_key: "bb17d0c4444616f24ab628eb30b4c196"
server_key_iv: "d3a3359085e2b0f7a77c34b7c2eb86bb"
server_key_version: "encryptionkey@0"
I20260812 03:57:23.816992 32207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:57:23.817937 32223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:23.818171 32207 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:57:23.818284 32207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "df523d794bbd45b3b03c4954f34ee0c4"
format_stamp: "Formatted at 2026-08-12 03:57:23 on dist-test-slave-8tdl"
server_key: "bb17d0c4444616f24ab628eb30b4c196"
server_key_iv: "d3a3359085e2b0f7a77c34b7c2eb86bb"
server_key_version: "encryptionkey@0"
I20260812 03:57:23.818362 32207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:23.832469 32207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:23.835527 32207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:23.835752 32207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:23.836015 32207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:23.836414 32207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:23.836462 32207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:23.836508 32207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:23.836534 32207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:23.849977 32207 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:43581
I20260812 03:57:23.850162 32355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:43581 every 8 connection(s)
I20260812 03:57:23.850522 32207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:57:23.852033 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 32207
I20260812 03:57:23.852108 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:57:23.852285 31919 external_mini_cluster.cc:1428] Setting key 913dfaee6e6c3cd8609c02c11a9eebbc
Aug 12 03:57:23 dist-test-slave-8tdl krb5kdc[31927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507043, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:23.864151 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:23.851626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:56843 (local address 127.31.43.254:38559)
0812 03:57:23.851691 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:23.851693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:23.852041 (+ 348us) server_negotiation.cc:407] Connection header received
0812 03:57:23.853116 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:23.853117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:23.853130 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:23.853192 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:23.854521 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:23.855509 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:23.857408 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:23.857645 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:23.857710 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:23.861049 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:23.861051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:23.861052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:23.861060 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:23.862550 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:23.863072 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:23.863073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:23.863074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:23.863135 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:23.863403 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:23.863404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:23.863404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:23.863531 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:23.863604 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:23.863909 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 03:57:23.864052 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:57:23.864315 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:23.851591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:38559 (local address 127.31.43.194:56843)
0812 03:57:23.851829 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:23.851871 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:23.852963 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:23.853284 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:23.853288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:23.854051 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:23.854407 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:23.854413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:23.855682 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:23.855683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:23.857308 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:23.857312 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:23.857763 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:23.857765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:23.858015 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:23.858866 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:23.858879 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:23.860923 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:23.862659 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:23.862668 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:23.862693 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:23.862977 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:23.863204 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:23.863205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:23.863206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:23.863328 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:23.863606 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:23.863614 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:23.863769 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 03:57:23.863875 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 03:57:23.864063 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
I20260812 03:57:23.865099 32356 heartbeater.cc:347] Connected to a master server at 127.31.43.254:38559
I20260812 03:57:23.865224 32356 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:23.865398 32356 heartbeater.cc:511] Master 127.31.43.254:38559 requested a full tablet report, sending...
I20260812 03:57:23.865886 31975 ts_manager.cc:194] Registered new tserver with Master: df523d794bbd45b3b03c4954f34ee0c4 (127.31.43.194:43581)
I20260812 03:57:23.866369 31975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:56843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 03:57:23.889505 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:38559
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:24.022459 32362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:24.022775 32362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:24.022822 32362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:24.024334 32362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:24.024422 32362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:24.024472 32362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:24.026393 32362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:38559
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:24.026788 32362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:24.027082 32362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:24.035020 32370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:24.035071 32367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:24.035293 32362 server_base.cc:1034] running on GCE node
W20260812 03:57:24.035087 32368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:24.035614 32362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:24.035920 32362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:24.037084 32362 hybrid_clock.cc:648] HybridClock initialized: now 1786507044037053 us; error 44 us; skew 500 ppm
Aug 12 03:57:24 dist-test-slave-8tdl krb5kdc[31927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507044, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:24.040339 32362 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 03:57:24.040885 32362 webserver.cc:460] Webserver started at http://127.31.43.195:38399/ using document root <none> and password file <none>
I20260812 03:57:24.041198 32362 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:24.041276 32362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:24.041430 32362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:24.042737 32362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "5250a452ffb74da49dc2bdbab6dfdc44"
format_stamp: "Formatted at 2026-08-12 03:57:24 on dist-test-slave-8tdl"
server_key: "1b585bf8270c5357b2bcdda457ad2281"
server_key_iv: "339c64e3e5a9d15e344d3022bcff8785"
server_key_version: "encryptionkey@0"
I20260812 03:57:24.043192 32362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "5250a452ffb74da49dc2bdbab6dfdc44"
format_stamp: "Formatted at 2026-08-12 03:57:24 on dist-test-slave-8tdl"
server_key: "1b585bf8270c5357b2bcdda457ad2281"
server_key_iv: "339c64e3e5a9d15e344d3022bcff8785"
server_key_version: "encryptionkey@0"
I20260812 03:57:24.045145 32362 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:57:24.046240 32378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:24.046571 32362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:24.046752 32362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "5250a452ffb74da49dc2bdbab6dfdc44"
format_stamp: "Formatted at 2026-08-12 03:57:24 on dist-test-slave-8tdl"
server_key: "1b585bf8270c5357b2bcdda457ad2281"
server_key_iv: "339c64e3e5a9d15e344d3022bcff8785"
server_key_version: "encryptionkey@0"
I20260812 03:57:24.046849 32362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:24.065977 32362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:24.069597 32362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:24.069746 32362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:24.070241 32362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:24.070767 32362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:24.070820 32362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:24.070845 32362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:24.070858 32362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:24.088850 32362 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:46349
I20260812 03:57:24.089156 32505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:46349 every 8 connection(s)
I20260812 03:57:24.089552 32362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:57:24.097716 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 32362
I20260812 03:57:24.097855 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:57:24.098067 31919 external_mini_cluster.cc:1428] Setting key 317271d20d26797d9896f78e7d8708ab
Aug 12 03:57:24 dist-test-slave-8tdl krb5kdc[31927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507044, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:24.103075 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:24.090511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:33163 (local address 127.31.43.254:38559)
0812 03:57:24.090588 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:24.090590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:24.091078 (+ 488us) server_negotiation.cc:407] Connection header received
0812 03:57:24.091963 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:24.091965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:24.091994 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:24.092070 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:24.093371 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:24.094000 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:24.095313 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:24.095545 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:24.095632 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:24.098997 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:24.098998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:24.098999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:24.099009 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:24.100893 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:24.101539 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:24.101547 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:24.101548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:24.101681 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:24.102069 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:24.102070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:24.102070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:24.102273 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:24.102374 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:24.102800 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 03:57:24.102934 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:57:24.103397 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:24.090502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:38559 (local address 127.31.43.195:33163)
0812 03:57:24.090835 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:24.090904 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:24.091808 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:24.092187 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:24.092192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:24.092940 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:24.093226 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:24.093233 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:24.094153 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:24.094154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:24.095197 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:24.095201 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:24.095651 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:24.095652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:24.095924 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:24.096639 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:24.096650 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:24.098861 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:24.101007 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:24.101017 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:24.101020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:24.101405 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:24.101772 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:24.101773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:24.101773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:24.101986 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:24.102354 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:24.102370 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:24.102572 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 03:57:24.102752 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 03:57:24.102981 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":212,"threads_started":1}
I20260812 03:57:24.104048 32506 heartbeater.cc:347] Connected to a master server at 127.31.43.254:38559
I20260812 03:57:24.104207 32506 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:24.104439 32506 heartbeater.cc:511] Master 127.31.43.254:38559 requested a full tablet report, sending...
I20260812 03:57:24.104960 31975 ts_manager.cc:194] Registered new tserver with Master: 5250a452ffb74da49dc2bdbab6dfdc44 (127.31.43.195:46349)
I20260812 03:57:24.105402 31975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:33163
I20260812 03:57:24.111333 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:57:24 dist-test-slave-8tdl krb5kdc[31927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507044, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:57:24 dist-test-slave-8tdl krb5kdc[31927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507044, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.43.193@KRBTEST.COM
I20260812 03:57:24.147990 32515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:24.135254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37830 (local address 127.31.43.193:34361)
0812 03:57:24.135405 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:24.135410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:24.135423 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:57:24.135456 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:24.135458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:24.135476 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:24.135648 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:24.136792 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:24.137635 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:24.138768 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:24.139443 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:24.139531 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:24.143136 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:24.143158 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:24.143158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:24.143169 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:24.145539 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:24.146269 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:24.146274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:24.146275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:24.146377 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:24.146851 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:24.146853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:24.146853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:24.147185 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:24.147352 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:24.147609 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 03:57:24.147794 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
W20260812 03:57:24.148878 32099 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37830
Aug 12 03:57:24 dist-test-slave-8tdl krb5kdc[31927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:57:24 dist-test-slave-8tdl krb5kdc[31927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.193@KRBTEST.COM
I20260812 03:57:24.181061 32515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:24.171884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.31.43.193:34361)
0812 03:57:24.172025 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:24.172027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:24.172040 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:57:24.172295 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:24.172297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:24.172312 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:24.172378 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:24.173118 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:24.173668 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:24.174506 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:24.174855 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:24.174949 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:24.178112 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:24.178114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:24.178114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:24.178124 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:24.179765 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:24.180160 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:24.180161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:24.180161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:24.180224 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:24.180505 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:24.180505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:24.180505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:24.180632 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:24.180732 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:24.180890 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 03:57:24.180981 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:57:24.181922 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 32052
I20260812 03:57:24.192216 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 32207
I20260812 03:57:24.203331 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 32362
I20260812 03:57:24.215492 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 31942
2026-08-12T03:57:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2363 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:24 dist-test-slave-8tdl krb5kdc[32523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:24 dist-test-slave-8tdl krb5kdc[32523](info): set up 2 sockets
Aug 12 03:57:24 dist-test-slave-8tdl krb5kdc[32523](info): commencing operation
krb5kdc: starting...
W20260812 03:57:25.438607 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:57:25 dist-test-slave-8tdl krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:57:25.590492 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:39119
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:44413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:39119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:25.683697 32538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:25.683992 32538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:25.684037 32538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:25.685292 32538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:25.685370 32538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:25.685395 32538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:25.685415 32538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:25.685436 32538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:25.687160 32538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:44413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:39119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:39119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:25.687503 32538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:25.687775 32538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:25.693187 32544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:25.694175 32543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:25.694662 32538 server_base.cc:1034] running on GCE node
W20260812 03:57:25.694747 32546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:25.695042 32538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:25.695339 32538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:25.696573 32538 hybrid_clock.cc:648] HybridClock initialized: now 1786507045696562 us; error 42 us; skew 500 ppm
Aug 12 03:57:25 dist-test-slave-8tdl krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507045, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:25.699533 32538 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:57:25.700079 32538 webserver.cc:460] Webserver started at http://127.31.43.254:39375/ using document root <none> and password file <none>
I20260812 03:57:25.700345 32538 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:25.700417 32538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:25.700567 32538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:25.701695 32538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "1fbcd5dd83bf4c4a8d18373e1d573a89"
format_stamp: "Formatted at 2026-08-12 03:57:25 on dist-test-slave-8tdl"
server_key: "dd1a8e269b0108cae5aa033dbf022e7f"
server_key_iv: "fac0844e597efb0b7f1b52a4735aca13"
server_key_version: "encryptionkey@0"
I20260812 03:57:25.702081 32538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "1fbcd5dd83bf4c4a8d18373e1d573a89"
format_stamp: "Formatted at 2026-08-12 03:57:25 on dist-test-slave-8tdl"
server_key: "dd1a8e269b0108cae5aa033dbf022e7f"
server_key_iv: "fac0844e597efb0b7f1b52a4735aca13"
server_key_version: "encryptionkey@0"
I20260812 03:57:25.703822 32538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:57:25.704777 32553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:25.704979 32538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:25.705097 32538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "1fbcd5dd83bf4c4a8d18373e1d573a89"
format_stamp: "Formatted at 2026-08-12 03:57:25 on dist-test-slave-8tdl"
server_key: "dd1a8e269b0108cae5aa033dbf022e7f"
server_key_iv: "fac0844e597efb0b7f1b52a4735aca13"
server_key_version: "encryptionkey@0"
I20260812 03:57:25.705180 32538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:25.742383 32538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:25.745009 32538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:25.745193 32538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:25.749675 32538 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:39119
I20260812 03:57:25.749742 32612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:39119 every 8 connection(s)
I20260812 03:57:25.750118 32538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:25.751202 32613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:25.753695 32613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89: Bootstrap starting.
I20260812 03:57:25.754462 32613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:25.754846 32613 log.cc:824] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:25.755748 32613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89: No bootstrap required, opened a new log
I20260812 03:57:25.757303 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 32538
I20260812 03:57:25.757382 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:25.757536 31919 external_mini_cluster.cc:1428] Setting key f730a40cb12b22e0cf80291795280455
I20260812 03:57:25.757624 32613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fbcd5dd83bf4c4a8d18373e1d573a89" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 39119 } }
I20260812 03:57:25.757823 32613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:25.757895 32613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fbcd5dd83bf4c4a8d18373e1d573a89, State: Initialized, Role: FOLLOWER
I20260812 03:57:25.758062 32613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fbcd5dd83bf4c4a8d18373e1d573a89" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 39119 } }
I20260812 03:57:25.758159 32613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:25.758193 32613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:25.758250 32613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:25.758949 32613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fbcd5dd83bf4c4a8d18373e1d573a89" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 39119 } }
I20260812 03:57:25.759047 32613 leader_election.cc:302] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fbcd5dd83bf4c4a8d18373e1d573a89; no voters:
I20260812 03:57:25.759268 32613 leader_election.cc:288] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:25.759295 32618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:25.759416 32618 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [term 1 LEADER]: Becoming Leader. State: Replica: 1fbcd5dd83bf4c4a8d18373e1d573a89, State: Running, Role: LEADER
I20260812 03:57:25.759503 32618 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fbcd5dd83bf4c4a8d18373e1d573a89" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 39119 } }
I20260812 03:57:25.759562 32613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:25.759879 32619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fbcd5dd83bf4c4a8d18373e1d573a89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fbcd5dd83bf4c4a8d18373e1d573a89" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 39119 } } }
I20260812 03:57:25.759884 32620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fbcd5dd83bf4c4a8d18373e1d573a89. Latest consensus state: current_term: 1 leader_uuid: "1fbcd5dd83bf4c4a8d18373e1d573a89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fbcd5dd83bf4c4a8d18373e1d573a89" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 39119 } } }
I20260812 03:57:25.759981 32619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:25.760035 32620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:25.760973 32624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:25.761498 32624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:25.763401 32624 catalog_manager.cc:1347] Generated new cluster ID: 4dcb0051155d4ed0befff724fb44f6ba
I20260812 03:57:25.763468 32624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:57:25 dist-test-slave-8tdl krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:25.771261 32624 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:25.771780 32624 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:25.775708 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:25.758241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47292 (local address 127.31.43.254:39119)
0812 03:57:25.758466 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:25.758475 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:25.758502 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:57:25.759103 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:25.759111 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:25.759196 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:25.759361 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:25.760547 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:25.762019 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:25.763164 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:25.763791 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:25.763924 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:25.766855 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:25.766861 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:25.766861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:25.766876 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:25.769698 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:25.770175 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:25.770179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:25.770179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:25.770309 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:25.770549 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:25.770550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:25.770550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:25.770865 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:25.771234 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:25.775198 (+ 3964us) server_negotiation.cc:299] Negotiation successful
0812 03:57:25.775424 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260812 03:57:25.780809 32624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fbcd5dd83bf4c4a8d18373e1d573a89: Generated new TSK 0
I20260812 03:57:25.781071 32624 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 03:57:25.814742 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:39119
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:44413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:25.901690 32641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:25.901929 32641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:25.901958 32641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:25.903256 32641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:25.903308 32641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:25.903338 32641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:25.905000 32641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:44413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:39119
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:25.905274 32641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:25.905525 32641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:25.911345 32647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:25.911350 32650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:25.911345 32648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:25.911399 32641 server_base.cc:1034] running on GCE node
I20260812 03:57:25.911953 32641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:25.912285 32641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:25.913470 32641 hybrid_clock.cc:648] HybridClock initialized: now 1786507045913446 us; error 41 us; skew 500 ppm
Aug 12 03:57:25 dist-test-slave-8tdl krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507045, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:25.916323 32641 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 03:57:25.916870 32641 webserver.cc:460] Webserver started at http://127.31.43.193:40393/ using document root <none> and password file <none>
I20260812 03:57:25.917141 32641 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:25.917213 32641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:25.917363 32641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:25.918501 32641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "845caf35f2c74f62a939b4f05736d5bb"
format_stamp: "Formatted at 2026-08-12 03:57:25 on dist-test-slave-8tdl"
server_key: "e115f36c09aa0fb736f9e5680a358a0f"
server_key_iv: "40d6d83c6c58d8de4b049f6716311b2e"
server_key_version: "encryptionkey@0"
I20260812 03:57:25.918992 32641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "845caf35f2c74f62a939b4f05736d5bb"
format_stamp: "Formatted at 2026-08-12 03:57:25 on dist-test-slave-8tdl"
server_key: "e115f36c09aa0fb736f9e5680a358a0f"
server_key_iv: "40d6d83c6c58d8de4b049f6716311b2e"
server_key_version: "encryptionkey@0"
I20260812 03:57:25.920473 32641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:25.921325 32660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:25.921528 32641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:25.921641 32641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "845caf35f2c74f62a939b4f05736d5bb"
format_stamp: "Formatted at 2026-08-12 03:57:25 on dist-test-slave-8tdl"
server_key: "e115f36c09aa0fb736f9e5680a358a0f"
server_key_iv: "40d6d83c6c58d8de4b049f6716311b2e"
server_key_version: "encryptionkey@0"
I20260812 03:57:25.921741 32641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:25.947836 32641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:25.950347 32641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:25.950552 32641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:25.950970 32641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:25.951349 32641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:25.951407 32641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:25.951450 32641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:25.951488 32641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:25.958040 32641 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:40791
I20260812 03:57:25.958109 321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:40791 every 8 connection(s)
I20260812 03:57:25.958529 32641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:57:25.961036 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 32641
I20260812 03:57:25.961151 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:57:25.961324 31919 external_mini_cluster.cc:1428] Setting key cb3fd9462380259d1cd3cf42201fa025
Aug 12 03:57:25 dist-test-slave-8tdl krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507045, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:25.970261 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:25.959487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:41177 (local address 127.31.43.254:39119)
0812 03:57:25.959541 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:25.959543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:25.959757 (+ 214us) server_negotiation.cc:407] Connection header received
0812 03:57:25.960998 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:25.961000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:25.961017 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:25.961107 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:25.962129 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:25.962648 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:25.963783 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:25.964075 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:25.964151 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:25.967016 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:25.967018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:25.967018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:25.967027 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:25.968553 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:25.969147 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:25.969148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:25.969148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:25.969212 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:25.969526 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:25.969527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:25.969527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:25.969653 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:25.969712 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:25.970094 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 03:57:25.970187 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:57:25.970551 324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:25.959412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:39119 (local address 127.31.43.193:41177)
0812 03:57:25.959588 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:25.959635 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:25.960877 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:25.961244 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:25.961249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:25.961723 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:25.962012 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:25.962019 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:25.962794 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:25.962795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:25.963664 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:25.963666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:25.964177 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:25.964179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:25.964344 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:25.964979 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:25.964991 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:25.966903 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:25.968651 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:25.968663 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:25.968666 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:25.969043 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:25.969293 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:25.969294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:25.969294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:25.969451 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:25.969722 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:25.969730 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:25.969911 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 03:57:25.970045 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 03:57:25.970253 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260812 03:57:25.971153 322 heartbeater.cc:347] Connected to a master server at 127.31.43.254:39119
I20260812 03:57:25.971259 322 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:25.971488 322 heartbeater.cc:511] Master 127.31.43.254:39119 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
I20260812 03:57:25.972158 32572 ts_manager.cc:194] Registered new tserver with Master: 845caf35f2c74f62a939b4f05736d5bb (127.31.43.193:40791)
I20260812 03:57:25.972831 32572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:41177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 03:57:25.994149 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:39119
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:44413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:26.082201 327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:26.082428 327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:26.082456 327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:26.083720 327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:26.083770 327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:26.083824 327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:26.085512 327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:44413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:39119
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:26.085773 327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:26.086035 327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:26.091380 332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:26.091368 336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:26.091368 334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:26.091754 327 server_base.cc:1034] running on GCE node
I20260812 03:57:26.091976 327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:26.092243 327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:26.093389 327 hybrid_clock.cc:648] HybridClock initialized: now 1786507046093376 us; error 32 us; skew 500 ppm
Aug 12 03:57:26 dist-test-slave-8tdl krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507046, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:26.096217 327 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 03:57:26.096819 327 webserver.cc:460] Webserver started at http://127.31.43.194:43079/ using document root <none> and password file <none>
I20260812 03:57:26.097095 327 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:26.097167 327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:26.097312 327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:26.098467 327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "2f66261b275d4549b3cc3d6f7d49e88c"
format_stamp: "Formatted at 2026-08-12 03:57:26 on dist-test-slave-8tdl"
server_key: "22682a30755a952704910487d4f02a06"
server_key_iv: "39700143874711ecc39458a6fd82710e"
server_key_version: "encryptionkey@0"
I20260812 03:57:26.098979 327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "2f66261b275d4549b3cc3d6f7d49e88c"
format_stamp: "Formatted at 2026-08-12 03:57:26 on dist-test-slave-8tdl"
server_key: "22682a30755a952704910487d4f02a06"
server_key_iv: "39700143874711ecc39458a6fd82710e"
server_key_version: "encryptionkey@0"
I20260812 03:57:26.100569 327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:57:26.101449 344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:26.101675 327 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:26.101791 327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "2f66261b275d4549b3cc3d6f7d49e88c"
format_stamp: "Formatted at 2026-08-12 03:57:26 on dist-test-slave-8tdl"
server_key: "22682a30755a952704910487d4f02a06"
server_key_iv: "39700143874711ecc39458a6fd82710e"
server_key_version: "encryptionkey@0"
I20260812 03:57:26.101876 327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:26.113644 327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:26.115993 327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:26.116173 327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:26.116431 327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:26.116766 327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:26.116822 327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:26.116863 327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:26.116895 327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:26.123533 327 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:35547
I20260812 03:57:26.123577 474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:35547 every 8 connection(s)
I20260812 03:57:26.123940 327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:57:26.130872 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 327
I20260812 03:57:26.131012 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
Aug 12 03:57:26 dist-test-slave-8tdl krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507046, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:26.131214 31919 external_mini_cluster.cc:1428] Setting key 0842001a5f70bf0d2ebb2eadfeda002c
I20260812 03:57:26.135042 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.124736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:37817 (local address 127.31.43.254:39119)
0812 03:57:26.124806 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:26.124808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:26.125297 (+ 489us) server_negotiation.cc:407] Connection header received
0812 03:57:26.126345 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:26.126347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:26.126358 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:26.126414 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:26.127150 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.127667 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.128730 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.128924 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.129070 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.131730 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:26.131731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:26.131731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:26.131739 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:26.133267 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:26.133854 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:26.133856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:26.133856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:26.133931 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:26.134227 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:26.134228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:26.134229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:26.134402 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:26.134480 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:26.134804 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 03:57:26.134931 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:57:26.135170 477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.124739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:39119 (local address 127.31.43.194:37817)
0812 03:57:26.125165 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:26.125207 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:26.126246 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:26.126476 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:26.126479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:26.126781 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:26.126994 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.126999 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.127774 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.127775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.128639 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.128641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.129161 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.129162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.129290 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.129853 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:26.129864 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:26.131636 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:26.133356 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:26.133366 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:26.133369 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:26.133768 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:26.134018 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:26.134019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:26.134019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:26.134159 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:26.134478 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:26.134484 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:26.134624 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 03:57:26.134775 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 03:57:26.134940 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":109,"threads_started":1}
I20260812 03:57:26.135670 475 heartbeater.cc:347] Connected to a master server at 127.31.43.254:39119
I20260812 03:57:26.135789 475 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:26.136001 475 heartbeater.cc:511] Master 127.31.43.254:39119 requested a full tablet report, sending...
I20260812 03:57:26.136507 32572 ts_manager.cc:194] Registered new tserver with Master: 2f66261b275d4549b3cc3d6f7d49e88c (127.31.43.194:35547)
I20260812 03:57:26.136925 32572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:37817
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 03:57:26.163168 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:39119
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:44413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:26.252888 480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:26.253202 480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:26.253252 480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:26.254630 480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:26.254755 480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:26.254805 480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:26.256765 480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:44413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:39119
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:26.257043 480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:26.257254 480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:26.263232 486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:26.263232 489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:26.263247 487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:26.263470 480 server_base.cc:1034] running on GCE node
I20260812 03:57:26.263811 480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:26.264112 480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:26.265305 480 hybrid_clock.cc:648] HybridClock initialized: now 1786507046265284 us; error 38 us; skew 500 ppm
Aug 12 03:57:26 dist-test-slave-8tdl krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507046, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:26.268379 480 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 03:57:26.269021 480 webserver.cc:460] Webserver started at http://127.31.43.195:36587/ using document root <none> and password file <none>
I20260812 03:57:26.269295 480 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:26.269366 480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:26.269546 480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:26.270820 480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "8358080458cd4f0e9b25f9208aa6a818"
format_stamp: "Formatted at 2026-08-12 03:57:26 on dist-test-slave-8tdl"
server_key: "e040718453bbdb1c017097ec3789def4"
server_key_iv: "f2c2fd0c3cedc0fd8564ea94a2fe35d4"
server_key_version: "encryptionkey@0"
I20260812 03:57:26.271307 480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "8358080458cd4f0e9b25f9208aa6a818"
format_stamp: "Formatted at 2026-08-12 03:57:26 on dist-test-slave-8tdl"
server_key: "e040718453bbdb1c017097ec3789def4"
server_key_iv: "f2c2fd0c3cedc0fd8564ea94a2fe35d4"
server_key_version: "encryptionkey@0"
I20260812 03:57:26.272950 480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:57:26.273934 497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:26.274224 480 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:26.274346 480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "8358080458cd4f0e9b25f9208aa6a818"
format_stamp: "Formatted at 2026-08-12 03:57:26 on dist-test-slave-8tdl"
server_key: "e040718453bbdb1c017097ec3789def4"
server_key_iv: "f2c2fd0c3cedc0fd8564ea94a2fe35d4"
server_key_version: "encryptionkey@0"
I20260812 03:57:26.274437 480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:26.299237 480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:26.301810 480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:26.301956 480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:26.302176 480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:26.302477 480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:26.302541 480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:26.302567 480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:26.302579 480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:26.309159 480 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:34785
I20260812 03:57:26.309288 633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:34785 every 8 connection(s)
I20260812 03:57:26.309659 480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:57:26.309816 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 480
I20260812 03:57:26.309930 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:57:26.310122 31919 external_mini_cluster.cc:1428] Setting key ca6a5bae7991f1362b5abdc61da3f4de
Aug 12 03:57:26 dist-test-slave-8tdl krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507046, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:26.320648 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.310670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:39989 (local address 127.31.43.254:39119)
0812 03:57:26.310748 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:26.310750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:26.311094 (+ 344us) server_negotiation.cc:407] Connection header received
0812 03:57:26.312201 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:26.312202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:26.312214 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:26.312275 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:26.312932 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.313460 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.314740 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.314958 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.315025 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.317511 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:26.317513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:26.317513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:26.317521 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:26.319163 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:26.319640 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:26.319641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:26.319641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:26.319698 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:26.320040 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:26.320040 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:26.320041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:26.320169 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:26.320228 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:26.320506 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 03:57:26.320577 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:57:26.320919 636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.310728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:39119 (local address 127.31.43.195:39989)
0812 03:57:26.310945 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:26.311010 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:26.312058 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:26.312340 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:26.312344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:26.312629 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:26.312843 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.312848 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.313568 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.313569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.314616 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.314620 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.315060 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.315061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.315171 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.315777 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:26.315786 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:26.317413 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:26.319264 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:26.319272 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:26.319274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:26.319560 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:26.319789 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:26.319790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:26.319790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:26.319921 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:26.320235 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:26.320241 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:26.320375 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 03:57:26.320477 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 03:57:26.320619 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":97,"threads_started":1}
I20260812 03:57:26.321710 634 heartbeater.cc:347] Connected to a master server at 127.31.43.254:39119
I20260812 03:57:26.321858 634 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:26.322096 634 heartbeater.cc:511] Master 127.31.43.254:39119 requested a full tablet report, sending...
I20260812 03:57:26.322516 32572 ts_manager.cc:194] Registered new tserver with Master: 8358080458cd4f0e9b25f9208aa6a818 (127.31.43.195:34785)
I20260812 03:57:26.322961 32572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:39989
I20260812 03:57:26.333398 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:57:26.342736 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.335880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47296 (local address 127.31.43.254:39119)
0812 03:57:26.335975 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:26.335977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:26.336061 (+ 84us) server_negotiation.cc:407] Connection header received
0812 03:57:26.336201 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:26.336202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:26.336213 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:26.336278 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:26.336908 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.337416 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.338201 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.338408 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.338524 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.339906 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:26.339907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:26.339908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:26.339915 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:26.341401 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:26.341743 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:26.341744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:26.341745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:26.341800 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:26.342007 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:26.342009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:26.342009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:26.342187 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:26.342279 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:26.342395 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 03:57:26.342562 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:57:26.346036 32572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:26.348335 32572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:26.356314 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.351588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:34785 (local address 127.0.0.1:55692)
0812 03:57:26.352084 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:26.352094 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:26.352148 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:26.352404 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:26.352404 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:26.352407 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:26.352459 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.352462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.353631 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.353632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.355092 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.355096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.355969 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.355970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.356059 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.356073 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:57:26.356162 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:57:26.356225 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":33,"thread_start_us":100,"threads_started":1}
I20260812 03:57:26.356323 651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.351881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55692 (local address 127.31.43.195:34785)
0812 03:57:26.352076 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:26.352080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:26.352220 (+ 140us) server_negotiation.cc:407] Connection header received
0812 03:57:26.352271 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:26.352274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:26.352290 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:26.352397 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:26.352523 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.353525 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.355206 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.355850 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.355955 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.356026 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:26.356195 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 03:57:26.356239 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":81,"threads_started":1}
I20260812 03:57:26.356755 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.351689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54108 (local address 127.31.43.194:35547)
0812 03:57:26.351846 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:26.351852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:26.351865 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:57:26.351995 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:26.351997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:26.352014 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:26.352108 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:26.352474 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.353802 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.355468 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.356294 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.356355 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.356411 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:26.356577 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 03:57:26.356676 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260812 03:57:26.356575 647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.351853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:40791 (local address 127.0.0.1:34572)
0812 03:57:26.352126 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:26.352138 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:26.352188 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:26.352657 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:26.352658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:26.352674 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:26.352722 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.352723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.353956 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.353957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.355147 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.355149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.356293 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.356295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.356369 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.356374 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:26.356405 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 03:57:26.356455 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":59,"thread_start_us":78,"threads_started":1}
I20260812 03:57:26.356906 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.352140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34572 (local address 127.31.43.193:40791)
0812 03:57:26.352333 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:26.352337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:26.352347 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:26.352497 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:26.352499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:26.352522 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:26.352662 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:26.352806 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.353856 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.355237 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.356171 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.356662 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.356738 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:26.356765 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 03:57:26.356812 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260812 03:57:26.356683 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.351550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:35547 (local address 127.0.0.1:54108)
0812 03:57:26.351750 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:26.351807 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:26.351875 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:26.352220 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:26.352223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:26.352225 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:26.352306 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.352308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.353934 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.353935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.355321 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.355324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.356443 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.356444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.356492 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.356496 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:26.356551 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:57:26.356586 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":81,"thread_start_us":101,"threads_started":1}
I20260812 03:57:26.357838 32714 tablet_service.cc:1467] Processing CreateTablet for tablet def2c754dbb6431493b23f2d39922feb (DEFAULT_TABLE table=test-table [id=3240b9df79ab4b9682bc051e4498b445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:26.357831 551 tablet_service.cc:1467] Processing CreateTablet for tablet def2c754dbb6431493b23f2d39922feb (DEFAULT_TABLE table=test-table [id=3240b9df79ab4b9682bc051e4498b445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:26.358208 551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet def2c754dbb6431493b23f2d39922feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:26.358213 32714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet def2c754dbb6431493b23f2d39922feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:26.359236 397 tablet_service.cc:1467] Processing CreateTablet for tablet def2c754dbb6431493b23f2d39922feb (DEFAULT_TABLE table=test-table [id=3240b9df79ab4b9682bc051e4498b445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:26.359640 397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet def2c754dbb6431493b23f2d39922feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:26.361824 655 tablet_bootstrap.cc:492] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb: Bootstrap starting.
I20260812 03:57:26.362140 656 tablet_bootstrap.cc:492] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818: Bootstrap starting.
I20260812 03:57:26.362593 655 tablet_bootstrap.cc:654] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:26.362869 656 tablet_bootstrap.cc:654] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:26.363372 655 log.cc:824] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:26.363467 656 log.cc:824] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:26.363727 657 tablet_bootstrap.cc:492] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c: Bootstrap starting.
I20260812 03:57:26.364452 657 tablet_bootstrap.cc:654] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:26.364689 656 tablet_bootstrap.cc:492] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818: No bootstrap required, opened a new log
I20260812 03:57:26.364770 656 ts_tablet_manager.cc:1397] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:57:26.364846 655 tablet_bootstrap.cc:492] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb: No bootstrap required, opened a new log
I20260812 03:57:26.364917 655 ts_tablet_manager.cc:1397] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:57:26.365034 657 log.cc:824] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:26.365749 657 tablet_bootstrap.cc:492] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c: No bootstrap required, opened a new log
I20260812 03:57:26.365831 657 ts_tablet_manager.cc:1397] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:57:26.366907 656 raft_consensus.cc:348] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f66261b275d4549b3cc3d6f7d49e88c" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35547 } } peers { permanent_uuid: "845caf35f2c74f62a939b4f05736d5bb" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40791 } } peers { permanent_uuid: "8358080458cd4f0e9b25f9208aa6a818" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34785 } }
I20260812 03:57:26.366961 655 raft_consensus.cc:348] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f66261b275d4549b3cc3d6f7d49e88c" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35547 } } peers { permanent_uuid: "845caf35f2c74f62a939b4f05736d5bb" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40791 } } peers { permanent_uuid: "8358080458cd4f0e9b25f9208aa6a818" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34785 } }
I20260812 03:57:26.367125 656 raft_consensus.cc:374] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:26.367125 655 raft_consensus.cc:374] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:26.367166 656 raft_consensus.cc:729] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8358080458cd4f0e9b25f9208aa6a818, State: Initialized, Role: FOLLOWER
I20260812 03:57:26.367166 655 raft_consensus.cc:729] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 845caf35f2c74f62a939b4f05736d5bb, State: Initialized, Role: FOLLOWER
I20260812 03:57:26.367312 656 consensus_queue.cc:260] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f66261b275d4549b3cc3d6f7d49e88c" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35547 } } peers { permanent_uuid: "845caf35f2c74f62a939b4f05736d5bb" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40791 } } peers { permanent_uuid: "8358080458cd4f0e9b25f9208aa6a818" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34785 } }
I20260812 03:57:26.367297 655 consensus_queue.cc:260] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f66261b275d4549b3cc3d6f7d49e88c" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35547 } } peers { permanent_uuid: "845caf35f2c74f62a939b4f05736d5bb" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40791 } } peers { permanent_uuid: "8358080458cd4f0e9b25f9208aa6a818" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34785 } }
I20260812 03:57:26.367632 655 ts_tablet_manager.cc:1428] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:57:26.367632 656 ts_tablet_manager.cc:1428] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:57:26.367511 657 raft_consensus.cc:348] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f66261b275d4549b3cc3d6f7d49e88c" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35547 } } peers { permanent_uuid: "845caf35f2c74f62a939b4f05736d5bb" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40791 } } peers { permanent_uuid: "8358080458cd4f0e9b25f9208aa6a818" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34785 } }
I20260812 03:57:26.367736 657 raft_consensus.cc:374] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:26.367780 657 raft_consensus.cc:729] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f66261b275d4549b3cc3d6f7d49e88c, State: Initialized, Role: FOLLOWER
I20260812 03:57:26.367787 322 heartbeater.cc:503] Master 127.31.43.254:39119 was elected leader, sending a full tablet report...
I20260812 03:57:26.367898 657 consensus_queue.cc:260] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f66261b275d4549b3cc3d6f7d49e88c" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35547 } } peers { permanent_uuid: "845caf35f2c74f62a939b4f05736d5bb" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40791 } } peers { permanent_uuid: "8358080458cd4f0e9b25f9208aa6a818" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34785 } }
I20260812 03:57:26.368288 657 ts_tablet_manager.cc:1428] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:57:26.368336 475 heartbeater.cc:503] Master 127.31.43.254:39119 was elected leader, sending a full tablet report...
I20260812 03:57:26.368543 634 heartbeater.cc:503] Master 127.31.43.254:39119 was elected leader, sending a full tablet report...
W20260812 03:57:26.374553 476 tablet.cc:2369] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:57:26.422891 662 raft_consensus.cc:482] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:26.423056 662 raft_consensus.cc:504] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f66261b275d4549b3cc3d6f7d49e88c" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35547 } } peers { permanent_uuid: "845caf35f2c74f62a939b4f05736d5bb" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40791 } } peers { permanent_uuid: "8358080458cd4f0e9b25f9208aa6a818" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34785 } }
I20260812 03:57:26.423629 662 leader_election.cc:288] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f66261b275d4549b3cc3d6f7d49e88c (127.31.43.194:35547), 845caf35f2c74f62a939b4f05736d5bb (127.31.43.193:40791)
I20260812 03:57:26.428198 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.424125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:55401 (local address 127.31.43.193:40791)
0812 03:57:26.424181 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:26.424184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:26.424314 (+ 130us) server_negotiation.cc:407] Connection header received
0812 03:57:26.424385 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:26.424387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:26.424400 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:26.424453 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:26.424668 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.425490 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.427138 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.427754 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.427838 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.427849 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:26.428071 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 03:57:26.428117 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:57:26.428198 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.424035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:58379 (local address 127.31.43.194:35547)
0812 03:57:26.424106 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:26.424108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:26.424120 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:57:26.424155 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:26.424156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:26.424168 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:26.424251 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:26.424498 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.425333 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.427148 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.427751 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.427839 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.427849 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:26.428070 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 03:57:26.428117 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:57:26.428303 636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.423866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:35547 (local address 127.31.43.195:58379)
0812 03:57:26.423926 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:26.423941 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:26.424012 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:26.424269 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:26.424270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:26.424272 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:26.424407 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.424409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.425433 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.425434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.427019 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.427023 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.427848 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.427849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.427987 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.427990 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:26.428054 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:57:26.428196 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":2}
I20260812 03:57:26.428437 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.424039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:40791 (local address 127.31.43.195:55401)
0812 03:57:26.424251 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:26.424264 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:26.424328 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:26.424483 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:26.424484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:26.424487 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:26.424557 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.424558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.425568 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.425569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.427058 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:26.427061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.427849 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.427849 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:26.427985 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.427990 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:26.428054 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:57:26.428196 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":97,"thread_start_us":154,"threads_started":1}
I20260812 03:57:26.428776 32737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "def2c754dbb6431493b23f2d39922feb" candidate_uuid: "8358080458cd4f0e9b25f9208aa6a818" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "845caf35f2c74f62a939b4f05736d5bb" is_pre_election: true
I20260812 03:57:26.428858 421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "def2c754dbb6431493b23f2d39922feb" candidate_uuid: "8358080458cd4f0e9b25f9208aa6a818" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f66261b275d4549b3cc3d6f7d49e88c" is_pre_election: true
I20260812 03:57:26.429006 32737 raft_consensus.cc:2454] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8358080458cd4f0e9b25f9208aa6a818 in term 0.
I20260812 03:57:26.429019 421 raft_consensus.cc:2454] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8358080458cd4f0e9b25f9208aa6a818 in term 0.
I20260812 03:57:26.429327 501 leader_election.cc:302] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f66261b275d4549b3cc3d6f7d49e88c, 8358080458cd4f0e9b25f9208aa6a818; no voters:
I20260812 03:57:26.429551 662 raft_consensus.cc:2781] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:26.429629 662 raft_consensus.cc:482] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:26.429651 662 raft_consensus.cc:3037] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:26.430968 662 raft_consensus.cc:504] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f66261b275d4549b3cc3d6f7d49e88c" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35547 } } peers { permanent_uuid: "845caf35f2c74f62a939b4f05736d5bb" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40791 } } peers { permanent_uuid: "8358080458cd4f0e9b25f9208aa6a818" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34785 } }
I20260812 03:57:26.431130 662 leader_election.cc:288] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [CANDIDATE]: Term 1 election: Requested vote from peers 2f66261b275d4549b3cc3d6f7d49e88c (127.31.43.194:35547), 845caf35f2c74f62a939b4f05736d5bb (127.31.43.193:40791)
I20260812 03:57:26.431370 32737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "def2c754dbb6431493b23f2d39922feb" candidate_uuid: "8358080458cd4f0e9b25f9208aa6a818" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "845caf35f2c74f62a939b4f05736d5bb"
I20260812 03:57:26.431457 421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "def2c754dbb6431493b23f2d39922feb" candidate_uuid: "8358080458cd4f0e9b25f9208aa6a818" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f66261b275d4549b3cc3d6f7d49e88c"
I20260812 03:57:26.431481 32737 raft_consensus.cc:3037] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:26.431566 421 raft_consensus.cc:3037] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:26.432330 32737 raft_consensus.cc:2454] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8358080458cd4f0e9b25f9208aa6a818 in term 1.
I20260812 03:57:26.432554 501 leader_election.cc:302] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8358080458cd4f0e9b25f9208aa6a818, 845caf35f2c74f62a939b4f05736d5bb; no voters:
I20260812 03:57:26.432668 421 raft_consensus.cc:2454] T def2c754dbb6431493b23f2d39922feb P 2f66261b275d4549b3cc3d6f7d49e88c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8358080458cd4f0e9b25f9208aa6a818 in term 1.
I20260812 03:57:26.432684 662 raft_consensus.cc:2781] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:26.432854 662 raft_consensus.cc:686] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [term 1 LEADER]: Becoming Leader. State: Replica: 8358080458cd4f0e9b25f9208aa6a818, State: Running, Role: LEADER
I20260812 03:57:26.432986 662 consensus_queue.cc:237] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f66261b275d4549b3cc3d6f7d49e88c" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35547 } } peers { permanent_uuid: "845caf35f2c74f62a939b4f05736d5bb" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40791 } } peers { permanent_uuid: "8358080458cd4f0e9b25f9208aa6a818" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34785 } }
I20260812 03:57:26.434134 32571 catalog_manager.cc:5495] T def2c754dbb6431493b23f2d39922feb P 8358080458cd4f0e9b25f9208aa6a818 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8358080458cd4f0e9b25f9208aa6a818 (127.31.43.195). New cstate: current_term: 1 leader_uuid: "8358080458cd4f0e9b25f9208aa6a818" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f66261b275d4549b3cc3d6f7d49e88c" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "845caf35f2c74f62a939b4f05736d5bb" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8358080458cd4f0e9b25f9208aa6a818" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34785 } health_report { overall_health: HEALTHY } } }
Aug 12 03:57:26 dist-test-slave-8tdl krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.193@KRBTEST.COM
I20260812 03:57:26.455509 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.446502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34576 (local address 127.31.43.193:40791)
0812 03:57:26.446550 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:26.446553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:26.446849 (+ 296us) server_negotiation.cc:407] Connection header received
0812 03:57:26.447027 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:26.447029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:26.447044 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:26.447099 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:26.448001 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.448710 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.449513 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.449715 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.449776 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.452286 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:26.452299 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:26.452299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:26.452307 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:26.454276 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:26.454659 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:26.454662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:26.454662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:26.454804 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:26.455008 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:26.455008 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:26.455009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:26.455165 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:26.455291 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:26.455359 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 03:57:26.455436 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 03:57:26.459350 323 tablet.cc:2369] T def2c754dbb6431493b23f2d39922feb P 845caf35f2c74f62a939b4f05736d5bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 03:57:26 dist-test-slave-8tdl krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507046, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:57:26 dist-test-slave-8tdl krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507046, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.43.193@KRBTEST.COM
I20260812 03:57:26.483732 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.475817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.31.43.193:40791)
0812 03:57:26.475901 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:26.475903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:26.475963 (+ 60us) server_negotiation.cc:407] Connection header received
0812 03:57:26.476045 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:26.476046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:26.476056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:26.476116 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:26.476815 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.477487 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.478293 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.478510 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.478598 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.481101 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:26.481102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:26.481102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:26.481111 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:26.482366 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:26.482775 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:26.482777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:26.482777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:26.482841 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:26.483086 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:26.483087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:26.483087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:26.483203 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:26.483311 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:26.483520 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 03:57:26.483618 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 03:57:26 dist-test-slave-8tdl krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:57:26 dist-test-slave-8tdl krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.193@KRBTEST.COM
I20260812 03:57:26.509434 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:26.501365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34586 (local address 127.31.43.193:40791)
0812 03:57:26.501479 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:26.501482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:26.501537 (+ 55us) server_negotiation.cc:407] Connection header received
0812 03:57:26.501595 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:26.501597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:26.501608 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:26.501664 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:26.502565 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.503187 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.503959 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:26.504217 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:26.504290 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:26.506811 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:26.506813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:26.506813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:26.506821 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:26.508213 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:26.508606 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:26.508607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:26.508608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:26.508671 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:26.508867 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:26.508869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:26.508869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:26.509001 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:26.509104 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:26.509235 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 03:57:26.509324 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 03:57:26.510296 32693 tablet_service.cc:2908] Rejecting scan request for tablet def2c754dbb6431493b23f2d39922feb: Uninitialized: clean time has not yet been initialized
I20260812 03:57:26.511683 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 32641
I20260812 03:57:26.524394 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 327
I20260812 03:57:26.532570 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 480
I20260812 03:57:26.540797 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 32538
2026-08-12T03:57:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2322 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:26 dist-test-slave-8tdl krb5kdc[685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:26 dist-test-slave-8tdl krb5kdc[685](info): set up 2 sockets
Aug 12 03:57:26 dist-test-slave-8tdl krb5kdc[685](info): commencing operation
krb5kdc: starting...
W20260812 03:57:27.696902 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:57:27 dist-test-slave-8tdl krb5kdc[685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:57:27.839057 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40763
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:40763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:27.927160 699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:27.927425 699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:27.927465 699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:27.928696 699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:27.928773 699 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:57:27.928795 699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:27.928819 699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:27.928848 699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:27.928879 699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:27.930492 699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:40763
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:27.930847 699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:27.931119 699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:27.936408 710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:27.936460 705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:27.936573 699 server_base.cc:1034] running on GCE node
W20260812 03:57:27.936429 704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:27.937096 699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:27.937529 699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:27.938757 699 hybrid_clock.cc:648] HybridClock initialized: now 1786507047938724 us; error 40 us; skew 500 ppm
Aug 12 03:57:27 dist-test-slave-8tdl krb5kdc[685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507047, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:27.947193 699 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:57:27.947696 699 webserver.cc:460] Webserver started at http://127.31.43.254:35133/ using document root <none> and password file <none>
I20260812 03:57:27.947960 699 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:27.948011 699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:27.948132 699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:27.949378 699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "d87eedade95248ba89fab68e0ae6a592"
format_stamp: "Formatted at 2026-08-12 03:57:27 on dist-test-slave-8tdl"
server_key: "58612f63735b5457b5ea2fe788cc6ad8"
server_key_iv: "a8991ec12e697f8da2749a46a7724832"
server_key_version: "encryptionkey@0"
I20260812 03:57:27.949743 699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "d87eedade95248ba89fab68e0ae6a592"
format_stamp: "Formatted at 2026-08-12 03:57:27 on dist-test-slave-8tdl"
server_key: "58612f63735b5457b5ea2fe788cc6ad8"
server_key_iv: "a8991ec12e697f8da2749a46a7724832"
server_key_version: "encryptionkey@0"
I20260812 03:57:27.951315 699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:27.952186 717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:27.952406 699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:27.952541 699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "d87eedade95248ba89fab68e0ae6a592"
format_stamp: "Formatted at 2026-08-12 03:57:27 on dist-test-slave-8tdl"
server_key: "58612f63735b5457b5ea2fe788cc6ad8"
server_key_iv: "a8991ec12e697f8da2749a46a7724832"
server_key_version: "encryptionkey@0"
I20260812 03:57:27.952646 699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:27.995218 699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:27.997463 699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:27.997663 699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:28.003239 699 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:40763
I20260812 03:57:28.003283 787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:40763 every 8 connection(s)
I20260812 03:57:28.003665 699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:28.004382 788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:28.005826 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 699
I20260812 03:57:28.005910 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:28.006139 31919 external_mini_cluster.cc:1428] Setting key 724b054959717e7d9fc005cda2e640f2
I20260812 03:57:28.007299 788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592: Bootstrap starting.
I20260812 03:57:28.008149 788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:28.008492 788 log.cc:824] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:28.009411 788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592: No bootstrap required, opened a new log
I20260812 03:57:28.012176 788 raft_consensus.cc:348] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87eedade95248ba89fab68e0ae6a592" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40763 } }
I20260812 03:57:28.012359 788 raft_consensus.cc:374] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:28.012385 788 raft_consensus.cc:729] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d87eedade95248ba89fab68e0ae6a592, State: Initialized, Role: FOLLOWER
I20260812 03:57:28.012529 788 consensus_queue.cc:260] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87eedade95248ba89fab68e0ae6a592" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40763 } }
I20260812 03:57:28.012626 788 raft_consensus.cc:388] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:28.012652 788 raft_consensus.cc:482] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:28.012722 788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:28.013358 788 raft_consensus.cc:504] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87eedade95248ba89fab68e0ae6a592" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40763 } }
I20260812 03:57:28.013487 788 leader_election.cc:302] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d87eedade95248ba89fab68e0ae6a592; no voters:
I20260812 03:57:28.013710 788 leader_election.cc:288] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:28.013756 794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:28.013877 794 raft_consensus.cc:686] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [term 1 LEADER]: Becoming Leader. State: Replica: d87eedade95248ba89fab68e0ae6a592, State: Running, Role: LEADER
Aug 12 03:57:28 dist-test-slave-8tdl krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:28.014021 788 sys_catalog.cc:564] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:28.014032 794 consensus_queue.cc:237] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87eedade95248ba89fab68e0ae6a592" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40763 } }
I20260812 03:57:28.014761 795 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d87eedade95248ba89fab68e0ae6a592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87eedade95248ba89fab68e0ae6a592" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40763 } } }
I20260812 03:57:28.014882 795 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:28.014767 796 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d87eedade95248ba89fab68e0ae6a592. Latest consensus state: current_term: 1 leader_uuid: "d87eedade95248ba89fab68e0ae6a592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87eedade95248ba89fab68e0ae6a592" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40763 } } }
I20260812 03:57:28.015106 796 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:28.015364 799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:28.016170 799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:28.019927 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.006829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55916 (local address 127.31.43.254:40763)
0812 03:57:28.007049 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.007058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.007103 (+ 45us) server_negotiation.cc:407] Connection header received
0812 03:57:28.007696 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.007708 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.007782 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.007939 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:28.008730 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.010010 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.010814 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.011507 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.011645 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.014619 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:28.014624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:28.014625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:28.014638 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:28.018031 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.018477 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.018480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.018481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.018580 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.018851 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.018852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.018852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.019148 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:28.019294 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.019547 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.019698 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":106,"threads_started":1}
I20260812 03:57:28.019930 799 catalog_manager.cc:1347] Generated new cluster ID: 257a861d4baa4c95930b4630a1c6e243
I20260812 03:57:28.020016 799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:28.044445 799 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:28.045032 799 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:28.056955 799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592: Generated new TSK 0
I20260812 03:57:28.057183 799 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 03:57:28.090024 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40763
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:28.178799 821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:28.179049 821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:28.179076 821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:28.180421 821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:28.180480 821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:28.180510 821 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:57:28.180529 821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:28.182175 821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.43.254:40763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:28.182467 821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:28.182771 821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:28.188150 828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:28.188187 830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:28.188413 826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:28.188657 821 server_base.cc:1034] running on GCE node
I20260812 03:57:28.188928 821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:28.189215 821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:28.190388 821 hybrid_clock.cc:648] HybridClock initialized: now 1786507048190379 us; error 31 us; skew 500 ppm
Aug 12 03:57:28 dist-test-slave-8tdl krb5kdc[685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507048, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:28.193109 821 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 03:57:28.193595 821 webserver.cc:460] Webserver started at http://127.31.43.193:46439/ using document root <none> and password file <none>
I20260812 03:57:28.193886 821 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:28.193964 821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:28.194128 821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:28.195413 821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "e76980dfaa4449748494e7f435770779"
format_stamp: "Formatted at 2026-08-12 03:57:28 on dist-test-slave-8tdl"
server_key: "695cfe5e9d0665f0f893245f242fb5ae"
server_key_iv: "56e95bae7e5a5397ec3d305175cda054"
server_key_version: "encryptionkey@0"
I20260812 03:57:28.195879 821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "e76980dfaa4449748494e7f435770779"
format_stamp: "Formatted at 2026-08-12 03:57:28 on dist-test-slave-8tdl"
server_key: "695cfe5e9d0665f0f893245f242fb5ae"
server_key_iv: "56e95bae7e5a5397ec3d305175cda054"
server_key_version: "encryptionkey@0"
I20260812 03:57:28.197461 821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:57:28.198307 837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:28.198491 821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:28.198617 821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "e76980dfaa4449748494e7f435770779"
format_stamp: "Formatted at 2026-08-12 03:57:28 on dist-test-slave-8tdl"
server_key: "695cfe5e9d0665f0f893245f242fb5ae"
server_key_iv: "56e95bae7e5a5397ec3d305175cda054"
server_key_version: "encryptionkey@0"
I20260812 03:57:28.198722 821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:28.221637 821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:28.224094 821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:28.224300 821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:28.224959 821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:28.225032 821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:28.225090 821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:28.225114 821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:28.232385 821 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:45371
I20260812 03:57:28.232471 966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:45371 every 8 connection(s)
Aug 12 03:57:28 dist-test-slave-8tdl krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507048, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:28.232805 821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:57:28.236146 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.225702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:44525 (local address 127.31.43.254:40763)
0812 03:57:28.225764 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.225766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.226056 (+ 290us) server_negotiation.cc:407] Connection header received
0812 03:57:28.226731 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.226732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.226746 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.226800 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:28.227680 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.228562 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.230254 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.230484 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.230566 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.233028 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:28.233030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:28.233030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:28.233039 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:28.234585 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.235102 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.235104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.235104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.235174 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.235431 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.235432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.235433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.235564 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:28.235643 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.235935 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.236050 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:57:28.236310 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.225716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40763 (local address 127.31.43.193:44525)
0812 03:57:28.225887 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.225925 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.226564 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.226883 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.226888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.227281 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:28.227588 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.227595 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.228690 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.228691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.230153 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.230157 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.230574 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.230576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.230765 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.231352 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:28.231361 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:28.232919 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:28.234712 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.234720 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.234723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.235014 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.235243 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.235244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.235244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.235361 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.235633 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:28.235641 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:28.235789 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.235895 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.236069 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260812 03:57:28.236368 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 821
I20260812 03:57:28.236501 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:57:28.236685 31919 external_mini_cluster.cc:1428] Setting key 4376d474b72c4fdad2b90e750e059f84
I20260812 03:57:28.237011 968 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40763
I20260812 03:57:28.237100 968 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:28.237311 968 heartbeater.cc:511] Master 127.31.43.254:40763 requested a full tablet report, sending...
I20260812 03:57:28.238024 734 ts_manager.cc:194] Registered new tserver with Master: e76980dfaa4449748494e7f435770779 (127.31.43.193:45371)
I20260812 03:57:28.238763 734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:44525
I20260812 03:57:28.245041 975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.237991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40763 (local address 127.31.43.193:36623)
0812 03:57:28.238145 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.238158 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.238210 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.238917 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.238918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.239163 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:28.239247 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.239249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.240012 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.240013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.240509 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.240510 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.241007 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.241008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.241078 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.241641 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:28.241645 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:28.241959 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:28.243973 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.243974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.243974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.244189 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.244424 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.244425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.244425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.244481 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.244772 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:28.244773 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:28.244818 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.244869 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.244964 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260812 03:57:28.245113 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.238055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:36623 (local address 127.31.43.254:40763)
0812 03:57:28.238797 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.238800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.238809 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:28.238830 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.238831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.238841 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.238899 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:28.239360 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.239911 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.240586 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.240880 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.240957 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.242058 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:28.242060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:28.242060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:28.242068 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:28.243854 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.244267 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.244269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.244270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.244342 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.244546 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.244547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.244547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.244703 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:28.244779 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.244922 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.245011 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719}
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 03:57:28.270501 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40763
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:28.360316 979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:28.360571 979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:28.360600 979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:28.361866 979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:28.361913 979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:28.361938 979 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:57:28.361958 979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:28.363519 979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.43.254:40763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:28.363736 979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:28.363950 979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:28.369586 985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:28.369647 987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:28.369612 984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:28.369879 979 server_base.cc:1034] running on GCE node
I20260812 03:57:28.370249 979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:28.370518 979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:28.371695 979 hybrid_clock.cc:648] HybridClock initialized: now 1786507048371680 us; error 34 us; skew 500 ppm
Aug 12 03:57:28 dist-test-slave-8tdl krb5kdc[685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507048, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:28.374794 979 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 03:57:28.375514 979 webserver.cc:460] Webserver started at http://127.31.43.194:39299/ using document root <none> and password file <none>
I20260812 03:57:28.375774 979 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:28.375824 979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:28.375998 979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:28.377185 979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "6a3917aae0744673980c4d3a5aec5fc3"
format_stamp: "Formatted at 2026-08-12 03:57:28 on dist-test-slave-8tdl"
server_key: "e9ba02119e59bcb1785d1e5814856f9b"
server_key_iv: "d0d863e9cbce62884c91fa62ff4d2d4b"
server_key_version: "encryptionkey@0"
I20260812 03:57:28.377599 979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "6a3917aae0744673980c4d3a5aec5fc3"
format_stamp: "Formatted at 2026-08-12 03:57:28 on dist-test-slave-8tdl"
server_key: "e9ba02119e59bcb1785d1e5814856f9b"
server_key_iv: "d0d863e9cbce62884c91fa62ff4d2d4b"
server_key_version: "encryptionkey@0"
I20260812 03:57:28.379315 979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 03:57:28.380250 996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:28.380446 979 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:28.380564 979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "6a3917aae0744673980c4d3a5aec5fc3"
format_stamp: "Formatted at 2026-08-12 03:57:28 on dist-test-slave-8tdl"
server_key: "e9ba02119e59bcb1785d1e5814856f9b"
server_key_iv: "d0d863e9cbce62884c91fa62ff4d2d4b"
server_key_version: "encryptionkey@0"
I20260812 03:57:28.380662 979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:28.417738 979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:28.420459 979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:28.420663 979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:28.421262 979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:28.421327 979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:28.421373 979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:28.421402 979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:57:28 dist-test-slave-8tdl krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507048, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:28.429950 979 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:45287
I20260812 03:57:28.430038 1128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:45287 every 8 connection(s)
I20260812 03:57:28.430401 979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:57:28.433229 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.421944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:59129 (local address 127.31.43.254:40763)
0812 03:57:28.422007 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.422010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.422298 (+ 288us) server_negotiation.cc:407] Connection header received
0812 03:57:28.422935 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.422937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.422951 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.423095 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:28.423870 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.424405 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.425542 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.425759 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.425840 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.429873 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:28.429875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:28.429876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:28.429886 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:28.431707 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.432177 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.432178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.432179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.432242 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.432530 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.432531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.432531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.432666 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:28.432739 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.433041 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.433133 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:57:28.433447 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.421937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40763 (local address 127.31.43.194:59129)
0812 03:57:28.422157 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.422198 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.422820 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.423217 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.423223 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.423523 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:28.423742 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.423747 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.424523 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.424524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.425412 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.425414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.425850 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.425852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.426031 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.426669 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:28.426708 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:28.429738 (+ 3030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:28.431829 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.431837 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.431839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.432090 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.432321 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.432322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.432323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.432451 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.432741 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:28.432749 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:28.432901 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.433016 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.433195 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":122,"threads_started":1}
I20260812 03:57:28.433966 1130 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40763
I20260812 03:57:28.434121 1130 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:28.434316 1130 heartbeater.cc:511] Master 127.31.43.254:40763 requested a full tablet report, sending...
I20260812 03:57:28.434795 735 ts_manager.cc:194] Registered new tserver with Master: 6a3917aae0744673980c4d3a5aec5fc3 (127.31.43.194:45287)
I20260812 03:57:28.435359 735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:59129
I20260812 03:57:28.437870 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 979
I20260812 03:57:28.437981 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:57:28.438191 31919 external_mini_cluster.cc:1428] Setting key c390283bb473969b527734723eaf45b1
I20260812 03:57:28.441705 1137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.435086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40763 (local address 127.31.43.194:41507)
0812 03:57:28.435246 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.435257 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.435320 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.435466 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.435466 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.435665 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:28.435733 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.435735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.436615 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.436616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.437246 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.437249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.437652 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.437653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.437755 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.438362 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:28.438368 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:28.438833 (+ 465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:28.440560 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.440561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.440562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.440745 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.440958 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.440959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.440959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.441025 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.441341 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:28.441343 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:28.441394 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.441475 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.441595 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
I20260812 03:57:28.441701 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.435235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:41507 (local address 127.31.43.254:40763)
0812 03:57:28.435276 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.435278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.435303 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:57:28.435375 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.435376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.435384 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.435458 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:28.435808 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.436523 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.437315 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.437492 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.437546 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.438907 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:28.438908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:28.438908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:28.438916 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:28.440445 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.440817 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.440819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.440819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.440882 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.441095 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.441096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.441096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.441259 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:28.441337 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.441501 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.441599 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 03:57:28.471091 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40763
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:28.559095 1142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:28.559333 1142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:28.559361 1142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:28.560591 1142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:28.560642 1142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:28.560664 1142 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:57:28.560683 1142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:28.562180 1142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.43.254:40763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:28.562444 1142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:28.562758 1142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:28.568017 1149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:28.568243 1142 server_base.cc:1034] running on GCE node
W20260812 03:57:28.568073 1148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:28.569692 1151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:28.569985 1142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:28.570323 1142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:28.571507 1142 hybrid_clock.cc:648] HybridClock initialized: now 1786507048571496 us; error 34 us; skew 500 ppm
Aug 12 03:57:28 dist-test-slave-8tdl krb5kdc[685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507048, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:28.574291 1142 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 03:57:28.574990 1142 webserver.cc:460] Webserver started at http://127.31.43.195:35707/ using document root <none> and password file <none>
I20260812 03:57:28.575248 1142 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:28.575322 1142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:28.575498 1142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:28.576792 1142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "1d60300530c9415999215aceb165d23a"
format_stamp: "Formatted at 2026-08-12 03:57:28 on dist-test-slave-8tdl"
server_key: "73d2c2031166464f5dc26ced92329656"
server_key_iv: "bec253f966d7290ccd70edaf1b3a2b28"
server_key_version: "encryptionkey@0"
I20260812 03:57:28.577208 1142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "1d60300530c9415999215aceb165d23a"
format_stamp: "Formatted at 2026-08-12 03:57:28 on dist-test-slave-8tdl"
server_key: "73d2c2031166464f5dc26ced92329656"
server_key_iv: "bec253f966d7290ccd70edaf1b3a2b28"
server_key_version: "encryptionkey@0"
I20260812 03:57:28.578857 1142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 03:57:28.579772 1159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:28.580055 1142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:28.580180 1142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "1d60300530c9415999215aceb165d23a"
format_stamp: "Formatted at 2026-08-12 03:57:28 on dist-test-slave-8tdl"
server_key: "73d2c2031166464f5dc26ced92329656"
server_key_iv: "bec253f966d7290ccd70edaf1b3a2b28"
server_key_version: "encryptionkey@0"
I20260812 03:57:28.580265 1142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:28.596251 1142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:28.598953 1142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:28.599144 1142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:28.599712 1142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:28.599773 1142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:28.599834 1142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:28.599869 1142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:57:28 dist-test-slave-8tdl krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507048, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:28.607686 1142 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:44557
I20260812 03:57:28.607765 1292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:44557 every 8 connection(s)
I20260812 03:57:28.608160 1142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:57:28.610822 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.600505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:60093 (local address 127.31.43.254:40763)
0812 03:57:28.600568 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.600571 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.600770 (+ 199us) server_negotiation.cc:407] Connection header received
0812 03:57:28.601559 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.601560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.601576 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.601662 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:28.602643 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.603545 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.604661 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.605009 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.605083 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.607711 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:28.607712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:28.607713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:28.607720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:28.609270 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.609731 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.609733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.609733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.609798 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.610060 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.610061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.610061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.610196 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:28.610276 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.610595 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.610727 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:57:28.611016 1169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.600442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40763 (local address 127.31.43.195:60093)
0812 03:57:28.600641 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.600679 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.601400 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.601776 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.601780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.602199 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:28.602518 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.602526 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.603654 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.603655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.604547 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.604550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.605115 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.605116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.605222 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.605806 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:28.605816 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:28.607606 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:28.609381 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.609387 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.609390 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.609635 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.609867 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.609868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.609868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.609988 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.610267 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:28.610274 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:28.610438 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.610563 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.610776 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
I20260812 03:57:28.611589 1293 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40763
I20260812 03:57:28.611683 1293 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:28.611894 1293 heartbeater.cc:511] Master 127.31.43.254:40763 requested a full tablet report, sending...
I20260812 03:57:28.612352 734 ts_manager.cc:194] Registered new tserver with Master: 1d60300530c9415999215aceb165d23a (127.31.43.195:44557)
I20260812 03:57:28.612915 734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:60093
I20260812 03:57:28.617437 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 1142
I20260812 03:57:28.617585 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:57:28.617767 31919 external_mini_cluster.cc:1428] Setting key 59f8e8293b4c6c6577e846c7b818bc7c
I20260812 03:57:28.618767 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:57:28.619500 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.612801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:44065 (local address 127.31.43.254:40763)
0812 03:57:28.612836 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.612838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.612970 (+ 132us) server_negotiation.cc:407] Connection header received
0812 03:57:28.613052 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.613053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.613183 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.613293 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:28.613855 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.614355 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.615170 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.615381 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.615454 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.616506 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:28.616508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:28.616508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:28.616515 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:28.618091 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.618485 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.618486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.618487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.618550 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.618799 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.618799 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.618800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.619058 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:28.619122 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.619345 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.619430 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260812 03:57:28.619607 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.612733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40763 (local address 127.31.43.195:44065)
0812 03:57:28.612904 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.612914 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.612992 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.613485 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.613486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.613701 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:28.613757 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.613758 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.614454 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.614454 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.615084 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.615086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.615466 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.615467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.615553 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.616086 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:28.616092 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:28.616403 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:28.618180 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.618181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.618181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.618406 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.618621 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.618621 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.618622 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.618712 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.619133 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:28.619134 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:28.619224 (+ 90us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.619313 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.619438 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
Aug 12 03:57:28 dist-test-slave-8tdl krb5kdc[685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507048, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:57:28 dist-test-slave-8tdl krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507048, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:28.645153 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.637415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55920 (local address 127.31.43.254:40763)
0812 03:57:28.637489 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.637491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.637502 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:57:28.637537 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.637538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.637547 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.637609 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:28.638249 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.638839 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.639605 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.639804 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.639886 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.642387 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:28.642388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:28.642389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:28.642397 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:28.644023 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.644381 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.644382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.644383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.644443 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.644663 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.644664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.644665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.644798 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:28.644894 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.644994 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.645079 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:57:28.647078 734 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:28.648926 734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:28.656237 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.651947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35974 (local address 127.31.43.194:45287)
0812 03:57:28.652196 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.652202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.652216 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:57:28.652266 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.652268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.652284 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.652633 (+ 349us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:28.652721 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.653439 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.654940 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.655794 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.655898 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.655985 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.656118 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.656153 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":107,"threads_started":1}
I20260812 03:57:28.656234 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.651647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:45287 (local address 127.0.0.1:35974)
0812 03:57:28.651825 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.651872 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.651941 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.652425 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.652426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.652428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:28.652514 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.652516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.653537 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.653538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.654834 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.654837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.655900 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.655901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.655988 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.656003 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.656090 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.656145 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":115,"threads_started":1}
I20260812 03:57:28.656750 1315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.651653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:44557 (local address 127.0.0.1:35858)
0812 03:57:28.652156 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.652167 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.652213 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.652566 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.652567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.652569 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:28.652604 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.652606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.653890 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.653891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.654990 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.654992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.656503 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.656504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.656564 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.656568 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.656622 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.656681 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":118,"threads_started":1}
I20260812 03:57:28.657012 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.651647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45371 (local address 127.0.0.1:57296)
0812 03:57:28.651825 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.651872 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.651942 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.652379 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.652383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.652385 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:28.652494 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.652496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.654962 (+ 2466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.654963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.655690 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.655692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.656788 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.656789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.656850 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.656853 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.656900 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.656932 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":2,"thread_start_us":68,"threads_started":1}
I20260812 03:57:28.657037 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.652137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.31.43.195:44557)
0812 03:57:28.652361 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.652365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.652374 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:28.652411 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.652413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.652428 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.652502 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:28.652688 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.653796 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.655073 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.655780 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.655866 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.655954 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.656866 (+ 912us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.656920 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":99,"threads_started":1}
I20260812 03:57:28.657325 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.651933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57296 (local address 127.31.43.193:45371)
0812 03:57:28.652134 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.652139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.652152 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:57:28.652197 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.652200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.652216 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.652294 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:28.652582 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.653796 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.655778 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.656621 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.656687 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.657131 (+ 444us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.657162 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.657227 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":113,"threads_started":1}
I20260812 03:57:28.657189 1054 tablet_service.cc:1467] Processing CreateTablet for tablet deabfb46f3d8499fb17094c5eef1129a (DEFAULT_TABLE table=test-table [id=740c71cd769046dbaac880c3928503b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:28.657661 1054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deabfb46f3d8499fb17094c5eef1129a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:28.657938 891 tablet_service.cc:1467] Processing CreateTablet for tablet deabfb46f3d8499fb17094c5eef1129a (DEFAULT_TABLE table=test-table [id=740c71cd769046dbaac880c3928503b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:28.658242 891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deabfb46f3d8499fb17094c5eef1129a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:28.659440 1213 tablet_service.cc:1467] Processing CreateTablet for tablet deabfb46f3d8499fb17094c5eef1129a (DEFAULT_TABLE table=test-table [id=740c71cd769046dbaac880c3928503b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:28.659801 1213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deabfb46f3d8499fb17094c5eef1129a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:28.661295 1320 tablet_bootstrap.cc:492] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3: Bootstrap starting.
I20260812 03:57:28.662096 1321 tablet_bootstrap.cc:492] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779: Bootstrap starting.
I20260812 03:57:28.662403 1320 tablet_bootstrap.cc:654] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:28.662817 1320 log.cc:824] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:28.662878 1321 tablet_bootstrap.cc:654] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:28.663216 1321 log.cc:824] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:28.663810 1320 tablet_bootstrap.cc:492] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3: No bootstrap required, opened a new log
I20260812 03:57:28.663889 1320 ts_tablet_manager.cc:1397] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:57:28.664104 1321 tablet_bootstrap.cc:492] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779: No bootstrap required, opened a new log
I20260812 03:57:28.664179 1321 ts_tablet_manager.cc:1397] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:28.666172 1320 raft_consensus.cc:348] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } }
I20260812 03:57:28.666283 1321 raft_consensus.cc:348] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } }
I20260812 03:57:28.666396 1320 raft_consensus.cc:374] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:28.666433 1321 raft_consensus.cc:374] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:28.666437 1320 raft_consensus.cc:729] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3917aae0744673980c4d3a5aec5fc3, State: Initialized, Role: FOLLOWER
I20260812 03:57:28.666478 1321 raft_consensus.cc:729] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e76980dfaa4449748494e7f435770779, State: Initialized, Role: FOLLOWER
I20260812 03:57:28.666608 1320 consensus_queue.cc:260] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } }
I20260812 03:57:28.666617 1321 consensus_queue.cc:260] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } }
I20260812 03:57:28.666988 1321 ts_tablet_manager.cc:1428] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:57:28.666988 1320 ts_tablet_manager.cc:1428] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:57:28.667035 1130 heartbeater.cc:503] Master 127.31.43.254:40763 was elected leader, sending a full tablet report...
I20260812 03:57:28.667141 968 heartbeater.cc:503] Master 127.31.43.254:40763 was elected leader, sending a full tablet report...
I20260812 03:57:28.667148 1323 tablet_bootstrap.cc:492] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a: Bootstrap starting.
I20260812 03:57:28.668464 1323 tablet_bootstrap.cc:654] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:28.668817 1323 log.cc:824] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:28.669624 1323 tablet_bootstrap.cc:492] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a: No bootstrap required, opened a new log
I20260812 03:57:28.669690 1323 ts_tablet_manager.cc:1397] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 03:57:28.671419 1323 raft_consensus.cc:348] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } }
I20260812 03:57:28.671573 1323 raft_consensus.cc:374] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:28.671597 1323 raft_consensus.cc:729] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d60300530c9415999215aceb165d23a, State: Initialized, Role: FOLLOWER
I20260812 03:57:28.671748 1323 consensus_queue.cc:260] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } }
I20260812 03:57:28.672047 1323 ts_tablet_manager.cc:1428] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:57:28.672152 1293 heartbeater.cc:503] Master 127.31.43.254:40763 was elected leader, sending a full tablet report...
W20260812 03:57:28.681123 1131 tablet.cc:2369] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:57:28.733848 969 tablet.cc:2369] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:57:28.775270 1326 raft_consensus.cc:482] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:28.775432 1326 raft_consensus.cc:504] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } }
I20260812 03:57:28.775815 1326 leader_election.cc:288] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d60300530c9415999215aceb165d23a (127.31.43.195:44557), e76980dfaa4449748494e7f435770779 (127.31.43.193:45371)
I20260812 03:57:28.779023 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.776357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:56233 (local address 127.31.43.193:45371)
0812 03:57:28.776428 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.776431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.776440 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:28.776471 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.776472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.776482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.776569 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:28.776741 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.777234 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.778208 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.778596 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.778686 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.778716 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.778910 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.778955 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:57:28.779023 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.776102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45371 (local address 127.31.43.194:56233)
0812 03:57:28.776179 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.776191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.776249 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.776578 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.776579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.776581 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:28.776660 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.776662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.777321 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.777322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.778104 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.778107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.778711 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.778712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.778813 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.778817 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.778891 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.778935 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":4}
I20260812 03:57:28.779134 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.776199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:59621 (local address 127.31.43.195:44557)
0812 03:57:28.776268 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.776270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.776448 (+ 178us) server_negotiation.cc:407] Connection header received
0812 03:57:28.776508 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.776509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.776517 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.776589 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:28.776775 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.777280 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.778257 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.778647 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.778811 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.778821 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.778996 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.779040 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:57:28.779182 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.776103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:44557 (local address 127.31.43.194:59621)
0812 03:57:28.776361 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.776391 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.776446 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.776620 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.776622 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.776624 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:28.776707 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.776708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.777370 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.777371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.778160 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.778163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.778817 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.778818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.778912 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.778915 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.778977 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.779021 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":77,"thread_start_us":108,"threads_started":1}
I20260812 03:57:28.779505 916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deabfb46f3d8499fb17094c5eef1129a" candidate_uuid: "6a3917aae0744673980c4d3a5aec5fc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e76980dfaa4449748494e7f435770779" is_pre_election: true
I20260812 03:57:28.779570 1237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deabfb46f3d8499fb17094c5eef1129a" candidate_uuid: "6a3917aae0744673980c4d3a5aec5fc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d60300530c9415999215aceb165d23a" is_pre_election: true
I20260812 03:57:28.779683 916 raft_consensus.cc:2454] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a3917aae0744673980c4d3a5aec5fc3 in term 0.
I20260812 03:57:28.779755 1237 raft_consensus.cc:2454] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a3917aae0744673980c4d3a5aec5fc3 in term 0.
I20260812 03:57:28.779961 997 leader_election.cc:302] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a3917aae0744673980c4d3a5aec5fc3, e76980dfaa4449748494e7f435770779; no voters:
I20260812 03:57:28.780128 1326 raft_consensus.cc:2781] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:28.780220 1326 raft_consensus.cc:482] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:28.780259 1326 raft_consensus.cc:3037] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:28.781366 1326 raft_consensus.cc:504] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } }
I20260812 03:57:28.781571 1326 leader_election.cc:288] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [CANDIDATE]: Term 1 election: Requested vote from peers 1d60300530c9415999215aceb165d23a (127.31.43.195:44557), e76980dfaa4449748494e7f435770779 (127.31.43.193:45371)
I20260812 03:57:28.781713 1237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deabfb46f3d8499fb17094c5eef1129a" candidate_uuid: "6a3917aae0744673980c4d3a5aec5fc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d60300530c9415999215aceb165d23a"
I20260812 03:57:28.781737 916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deabfb46f3d8499fb17094c5eef1129a" candidate_uuid: "6a3917aae0744673980c4d3a5aec5fc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e76980dfaa4449748494e7f435770779"
I20260812 03:57:28.781812 1237 raft_consensus.cc:3037] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:28.781813 916 raft_consensus.cc:3037] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:28.782765 1237 raft_consensus.cc:2454] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a3917aae0744673980c4d3a5aec5fc3 in term 1.
I20260812 03:57:28.782766 916 raft_consensus.cc:2454] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a3917aae0744673980c4d3a5aec5fc3 in term 1.
I20260812 03:57:28.782954 997 leader_election.cc:302] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a3917aae0744673980c4d3a5aec5fc3, e76980dfaa4449748494e7f435770779; no voters:
I20260812 03:57:28.783077 1326 raft_consensus.cc:2781] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:28.783275 1326 raft_consensus.cc:686] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 1 LEADER]: Becoming Leader. State: Replica: 6a3917aae0744673980c4d3a5aec5fc3, State: Running, Role: LEADER
I20260812 03:57:28.783394 1326 consensus_queue.cc:237] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } }
I20260812 03:57:28.784467 735 catalog_manager.cc:5495] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a3917aae0744673980c4d3a5aec5fc3 (127.31.43.194). New cstate: current_term: 1 leader_uuid: "6a3917aae0744673980c4d3a5aec5fc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } health_report { overall_health: UNKNOWN } } }
I20260812 03:57:28.807919 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.804560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35976 (local address 127.31.43.194:45287)
0812 03:57:28.804629 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.804631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.804641 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:28.804692 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.804693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.804711 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.804817 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:28.805062 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.805693 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.806795 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.807064 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.807135 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.807489 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:28.807595 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:28.807664 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.807801 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.807829 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:57:28.810583 1237 raft_consensus.cc:1261] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a [term 1 FOLLOWER]: Refusing update from remote peer 6a3917aae0744673980c4d3a5aec5fc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:28.811101 1326 consensus_queue.cc:1035] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:28.812366 916 raft_consensus.cc:1261] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779 [term 1 FOLLOWER]: Refusing update from remote peer 6a3917aae0744673980c4d3a5aec5fc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:28.812716 1326 consensus_queue.cc:1035] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:28.814855 1335 mvcc.cc:205] Tried to move back new op lower bound from 7317532871923720192 to 7317532871817527296. Current Snapshot: MvccSnapshot[applied={T|T < 7317532871923720192}]
I20260812 03:57:28.815073 1336 mvcc.cc:205] Tried to move back new op lower bound from 7317532871923720192 to 7317532871817527296. Current Snapshot: MvccSnapshot[applied={T|T < 7317532871923720192 or (T in {7317532871923720192})}]
I20260812 03:57:28.816676 1338 mvcc.cc:205] Tried to move back new op lower bound from 7317532871923720192 to 7317532871817527296. Current Snapshot: MvccSnapshot[applied={T|T < 7317532871923720192}]
I20260812 03:57:28.819809 735 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:28.819973 735 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:28.820952 735 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592: Sending DeleteTablet for 3 replicas of tablet deabfb46f3d8499fb17094c5eef1129a
I20260812 03:57:28.821489 1213 tablet_service.cc:1514] Processing DeleteTablet for tablet deabfb46f3d8499fb17094c5eef1129a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:28 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:35858
I20260812 03:57:28.821482 891 tablet_service.cc:1514] Processing DeleteTablet for tablet deabfb46f3d8499fb17094c5eef1129a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:28 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:57296
I20260812 03:57:28.821785 1347 tablet_replica.cc:330] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a: stopping tablet replica
I20260812 03:57:28.821765 1054 tablet_service.cc:1514] Processing DeleteTablet for tablet deabfb46f3d8499fb17094c5eef1129a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:28 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:35974
I20260812 03:57:28.821918 1347 raft_consensus.cc:2229] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:28.822059 1347 raft_consensus.cc:2258] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:28.822088 1348 tablet_replica.cc:330] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3: stopping tablet replica
I20260812 03:57:28.822269 1348 raft_consensus.cc:2229] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:57:28.822459 1348 raft_consensus.cc:2258] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:28.822417 735 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:28.822582 735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:28.822813 1347 ts_tablet_manager.cc:1905] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:28.823009 1348 ts_tablet_manager.cc:1905] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:28.824499 891 tablet_service.cc:1467] Processing CreateTablet for tablet d17b2c19a7854270b35d82227b553011 (DEFAULT_TABLE table=test-table [id=cf5be45d9b414a5197d8c51c1cb620e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:28.824641 891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d17b2c19a7854270b35d82227b553011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:28.825224 1348 ts_tablet_manager.cc:1918] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:28.825229 1347 ts_tablet_manager.cc:1918] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:28.825304 1347 log.cc:1196] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/deabfb46f3d8499fb17094c5eef1129a
I20260812 03:57:28.825304 1348 log.cc:1196] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/deabfb46f3d8499fb17094c5eef1129a
I20260812 03:57:28.825613 1347 ts_tablet_manager.cc:1939] T deabfb46f3d8499fb17094c5eef1129a P 1d60300530c9415999215aceb165d23a: Deleting consensus metadata
I20260812 03:57:28.825616 1348 ts_tablet_manager.cc:1939] T deabfb46f3d8499fb17094c5eef1129a P 6a3917aae0744673980c4d3a5aec5fc3: Deleting consensus metadata
I20260812 03:57:28.826249 1321 tablet_bootstrap.cc:492] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779: Bootstrap starting.
I20260812 03:57:28.826318 720 catalog_manager.cc:4841] TS 6a3917aae0744673980c4d3a5aec5fc3 (127.31.43.194:45287): tablet deabfb46f3d8499fb17094c5eef1129a (table test-table [id=740c71cd769046dbaac880c3928503b7]) successfully deleted
I20260812 03:57:28.826598 1054 tablet_service.cc:1467] Processing CreateTablet for tablet d17b2c19a7854270b35d82227b553011 (DEFAULT_TABLE table=test-table [id=cf5be45d9b414a5197d8c51c1cb620e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:28.826735 1054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d17b2c19a7854270b35d82227b553011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:28.827041 1321 tablet_bootstrap.cc:654] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:28.827867 1321 tablet_bootstrap.cc:492] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779: No bootstrap required, opened a new log
I20260812 03:57:28.827925 1321 ts_tablet_manager.cc:1397] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:57:28.828102 1321 raft_consensus.cc:348] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.828192 1321 raft_consensus.cc:374] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:28.828217 1321 raft_consensus.cc:729] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e76980dfaa4449748494e7f435770779, State: Initialized, Role: FOLLOWER
I20260812 03:57:28.828266 1321 consensus_queue.cc:260] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.828279 721 catalog_manager.cc:4841] TS 1d60300530c9415999215aceb165d23a (127.31.43.195:44557): tablet deabfb46f3d8499fb17094c5eef1129a (table test-table [id=740c71cd769046dbaac880c3928503b7]) successfully deleted
I20260812 03:57:28.828393 1321 ts_tablet_manager.cc:1428] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 03:57:28.828510 1320 tablet_bootstrap.cc:492] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3: Bootstrap starting.
I20260812 03:57:28.828538 1213 tablet_service.cc:1467] Processing CreateTablet for tablet d17b2c19a7854270b35d82227b553011 (DEFAULT_TABLE table=test-table [id=cf5be45d9b414a5197d8c51c1cb620e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:28.828658 1213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d17b2c19a7854270b35d82227b553011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:28.828967 1320 tablet_bootstrap.cc:654] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:28.829020 1346 tablet_replica.cc:330] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779: stopping tablet replica
I20260812 03:57:28.829164 1346 raft_consensus.cc:2229] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:28.829290 1346 raft_consensus.cc:2258] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:28.829865 1346 ts_tablet_manager.cc:1905] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:28.830071 1323 tablet_bootstrap.cc:492] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a: Bootstrap starting.
I20260812 03:57:28.830533 1323 tablet_bootstrap.cc:654] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:28.831368 1320 tablet_bootstrap.cc:492] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3: No bootstrap required, opened a new log
I20260812 03:57:28.831445 1320 ts_tablet_manager.cc:1397] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:57:28.831661 1320 raft_consensus.cc:348] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.831748 1320 raft_consensus.cc:374] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:28.831768 1320 raft_consensus.cc:729] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3917aae0744673980c4d3a5aec5fc3, State: Initialized, Role: FOLLOWER
I20260812 03:57:28.831820 1320 consensus_queue.cc:260] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.831912 1320 ts_tablet_manager.cc:1428] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:28.832580 1346 ts_tablet_manager.cc:1918] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:28.832715 1346 log.cc:1196] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/wal/wals/deabfb46f3d8499fb17094c5eef1129a
I20260812 03:57:28.832942 1323 tablet_bootstrap.cc:492] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a: No bootstrap required, opened a new log
I20260812 03:57:28.833002 1323 ts_tablet_manager.cc:1397] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 03:57:28.833113 1346 ts_tablet_manager.cc:1939] T deabfb46f3d8499fb17094c5eef1129a P e76980dfaa4449748494e7f435770779: Deleting consensus metadata
I20260812 03:57:28.833181 1323 raft_consensus.cc:348] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.833261 1323 raft_consensus.cc:374] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:28.833282 1323 raft_consensus.cc:729] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d60300530c9415999215aceb165d23a, State: Initialized, Role: FOLLOWER
I20260812 03:57:28.833329 1323 consensus_queue.cc:260] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.833441 1323 ts_tablet_manager.cc:1428] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:28.833612 718 catalog_manager.cc:4841] TS e76980dfaa4449748494e7f435770779 (127.31.43.193:45371): tablet deabfb46f3d8499fb17094c5eef1129a (table test-table [id=740c71cd769046dbaac880c3928503b7]) successfully deleted
W20260812 03:57:28.858843 1294 tablet.cc:2369] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:57:28.902817 1325 raft_consensus.cc:482] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:28.902953 1325 raft_consensus.cc:504] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.903371 1325 leader_election.cc:288] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a3917aae0744673980c4d3a5aec5fc3 (127.31.43.194:45287), 1d60300530c9415999215aceb165d23a (127.31.43.195:44557)
I20260812 03:57:28.907573 1352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.903621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:44557 (local address 127.31.43.193:37985)
0812 03:57:28.903873 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.903884 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.903938 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.904235 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.904236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.904239 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:28.904444 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.904447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.905388 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.905389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.906361 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.906364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.907250 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.907251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.907334 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.907339 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.907413 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.907458 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":72,"threads_started":1}
I20260812 03:57:28.907572 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.903816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:37985 (local address 127.31.43.195:44557)
0812 03:57:28.903889 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.903892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.903927 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:57:28.904059 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.904060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.904073 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.904205 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:28.904547 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.905294 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.906467 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.907129 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.907233 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.907245 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.907433 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.907492 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:57:28.907666 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.903761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:50893 (local address 127.31.43.194:45287)
0812 03:57:28.903833 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.903835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.903848 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:57:28.903986 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.903987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.903999 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.904078 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:28.904294 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.905059 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.906546 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.907177 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.907245 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.907254 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.907531 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.907575 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:57:28.907925 1353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.903614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:45287 (local address 127.31.43.193:50893)
0812 03:57:28.903763 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.903778 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.903845 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.904084 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.904085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.904087 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:28.904181 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.904185 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.905177 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.905178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.906443 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.906446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.907319 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.907320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.907429 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.907433 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.907822 (+ 389us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.907866 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260812 03:57:28.908339 1237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d17b2c19a7854270b35d82227b553011" candidate_uuid: "e76980dfaa4449748494e7f435770779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d60300530c9415999215aceb165d23a" is_pre_election: true
I20260812 03:57:28.908464 1237 raft_consensus.cc:2454] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e76980dfaa4449748494e7f435770779 in term 0.
I20260812 03:57:28.908504 1078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d17b2c19a7854270b35d82227b553011" candidate_uuid: "e76980dfaa4449748494e7f435770779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a3917aae0744673980c4d3a5aec5fc3" is_pre_election: true
I20260812 03:57:28.908661 1078 raft_consensus.cc:2454] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e76980dfaa4449748494e7f435770779 in term 0.
I20260812 03:57:28.908916 840 leader_election.cc:302] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a3917aae0744673980c4d3a5aec5fc3, e76980dfaa4449748494e7f435770779; no voters:
I20260812 03:57:28.909098 1325 raft_consensus.cc:2781] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:28.909143 1325 raft_consensus.cc:482] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:28.909165 1325 raft_consensus.cc:3037] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:28.910142 1325 raft_consensus.cc:504] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.910322 1325 leader_election.cc:288] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [CANDIDATE]: Term 1 election: Requested vote from peers 6a3917aae0744673980c4d3a5aec5fc3 (127.31.43.194:45287), 1d60300530c9415999215aceb165d23a (127.31.43.195:44557)
I20260812 03:57:28.910522 1078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d17b2c19a7854270b35d82227b553011" candidate_uuid: "e76980dfaa4449748494e7f435770779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a3917aae0744673980c4d3a5aec5fc3"
I20260812 03:57:28.910558 1237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d17b2c19a7854270b35d82227b553011" candidate_uuid: "e76980dfaa4449748494e7f435770779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d60300530c9415999215aceb165d23a"
I20260812 03:57:28.910620 1078 raft_consensus.cc:3037] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:28.910631 1237 raft_consensus.cc:3037] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:28.911731 1237 raft_consensus.cc:2454] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e76980dfaa4449748494e7f435770779 in term 1.
I20260812 03:57:28.911955 1078 raft_consensus.cc:2454] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e76980dfaa4449748494e7f435770779 in term 1.
I20260812 03:57:28.912003 841 leader_election.cc:302] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d60300530c9415999215aceb165d23a, e76980dfaa4449748494e7f435770779; no voters:
I20260812 03:57:28.912122 1325 raft_consensus.cc:2781] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:28.912293 1325 raft_consensus.cc:686] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 1 LEADER]: Becoming Leader. State: Replica: e76980dfaa4449748494e7f435770779, State: Running, Role: LEADER
I20260812 03:57:28.912379 1325 consensus_queue.cc:237] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.913291 735 catalog_manager.cc:5495] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 reported cstate change: term changed from 0 to 1, leader changed from <none> to e76980dfaa4449748494e7f435770779 (127.31.43.193). New cstate: current_term: 1 leader_uuid: "e76980dfaa4449748494e7f435770779" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } health_report { overall_health: UNKNOWN } } }
Aug 12 03:57:28 dist-test-slave-8tdl krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507047, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:28.987669 1362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.979708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40763 (local address 127.0.0.1:55932)
0812 03:57:28.979986 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:28.979996 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:28.980057 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:28.980290 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:28.980291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:28.980631 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:28.980683 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.980685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.981419 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.981419 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.981961 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:28.981963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.982320 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.982321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:28.982370 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.983086 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:28.983092 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:28.984798 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:28.986456 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.986464 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.986466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.986759 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.986950 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:28.986950 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:28.986950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:28.987010 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:28.987303 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:28.987311 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:28.987362 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 03:57:28.987437 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:57:28.987549 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":123,"threads_started":1}
I20260812 03:57:28.987717 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:28.979810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55932 (local address 127.31.43.254:40763)
0812 03:57:28.979922 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:28.979924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:28.980045 (+ 121us) server_negotiation.cc:407] Connection header received
0812 03:57:28.980118 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:28.980119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:28.980129 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:28.980210 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:28.980793 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.981332 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.982052 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:28.982237 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:28.982297 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:28.984888 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:28.984889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:28.984890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:28.984896 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:28.986359 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.986833 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.986834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.986834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.986888 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:28.987074 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:28.987075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:28.987075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:28.987203 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:28.987260 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:28.987498 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 03:57:28.987607 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":81}
I20260812 03:57:28.990545 735 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:55932:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:57:28.990967 735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:28.992727 1213 tablet_service.cc:1467] Processing CreateTablet for tablet c5416029670c4ee28397d5d88f4d20cc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=04fb31c98cf847378b200983223a25af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:57:28.992831 891 tablet_service.cc:1467] Processing CreateTablet for tablet c5416029670c4ee28397d5d88f4d20cc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=04fb31c98cf847378b200983223a25af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:57:28.992821 1054 tablet_service.cc:1467] Processing CreateTablet for tablet c5416029670c4ee28397d5d88f4d20cc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=04fb31c98cf847378b200983223a25af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:57:28.993000 891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5416029670c4ee28397d5d88f4d20cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:28.993000 1054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5416029670c4ee28397d5d88f4d20cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:28.993007 1213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5416029670c4ee28397d5d88f4d20cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:28.994576 1323 tablet_bootstrap.cc:492] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: Bootstrap starting.
I20260812 03:57:28.994740 1320 tablet_bootstrap.cc:492] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3: Bootstrap starting.
I20260812 03:57:28.994820 1321 tablet_bootstrap.cc:492] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779: Bootstrap starting.
I20260812 03:57:28.995170 1323 tablet_bootstrap.cc:654] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:28.995172 1320 tablet_bootstrap.cc:654] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:28.995253 1321 tablet_bootstrap.cc:654] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:28.995857 1323 tablet_bootstrap.cc:492] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: No bootstrap required, opened a new log
I20260812 03:57:28.995916 1323 ts_tablet_manager.cc:1397] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:28.995997 1320 tablet_bootstrap.cc:492] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3: No bootstrap required, opened a new log
I20260812 03:57:28.996075 1320 ts_tablet_manager.cc:1397] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:28.996138 1323 raft_consensus.cc:348] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.996237 1323 raft_consensus.cc:374] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:28.996254 1323 raft_consensus.cc:729] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d60300530c9415999215aceb165d23a, State: Initialized, Role: FOLLOWER
I20260812 03:57:28.996250 1320 raft_consensus.cc:348] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.996338 1320 raft_consensus.cc:374] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:28.996318 1323 consensus_queue.cc:260] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.996361 1320 raft_consensus.cc:729] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3917aae0744673980c4d3a5aec5fc3, State: Initialized, Role: FOLLOWER
I20260812 03:57:28.996402 1320 consensus_queue.cc:260] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.996443 1323 ts_tablet_manager.cc:1428] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:28.996490 1320 ts_tablet_manager.cc:1428] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:57:28.996505 1328 tablet_replica.cc:439] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } }
I20260812 03:57:28.996621 1328 tablet_replica.cc:442] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:28.996629 1331 tablet_replica.cc:439] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } }
I20260812 03:57:28.996722 1331 tablet_replica.cc:442] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:28.996948 1321 tablet_bootstrap.cc:492] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779: No bootstrap required, opened a new log
I20260812 03:57:28.997012 1321 ts_tablet_manager.cc:1397] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:57:28.997215 1321 raft_consensus.cc:348] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.997298 1321 raft_consensus.cc:374] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:28.997323 1321 raft_consensus.cc:729] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e76980dfaa4449748494e7f435770779, State: Initialized, Role: FOLLOWER
I20260812 03:57:28.997376 1321 consensus_queue.cc:260] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:28.997483 1321 ts_tablet_manager.cc:1428] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:28.997543 1325 tablet_replica.cc:439] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } }
I20260812 03:57:28.997627 1325 tablet_replica.cc:442] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:29.230110 1328 raft_consensus.cc:482] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:29.230262 1328 raft_consensus.cc:504] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:29.230765 1328 leader_election.cc:288] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e76980dfaa4449748494e7f435770779 (127.31.43.193:45371), 6a3917aae0744673980c4d3a5aec5fc3 (127.31.43.194:45287)
I20260812 03:57:29.234488 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.231223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:43825 (local address 127.31.43.194:45287)
0812 03:57:29.231278 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:29.231280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:29.231287 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:57:29.231315 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:29.231316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:29.231324 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:29.231398 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:29.231633 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.232174 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.233731 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.234111 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.234203 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.234212 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:29.234369 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:57:29.234417 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:57:29.234509 1367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.231013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:45287 (local address 127.31.43.195:43825)
0812 03:57:29.231157 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:29.231168 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:29.231223 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:29.231452 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:29.231453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:29.231457 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:29.231538 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:29.231539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.232294 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.232295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:29.233607 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:29.233610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.234202 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.234203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:29.234281 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.234287 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:57:29.234362 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:57:29.234419 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":45,"thread_start_us":61,"threads_started":1}
I20260812 03:57:29.234766 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.231065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:54169 (local address 127.31.43.193:45371)
0812 03:57:29.231127 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:29.231130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:29.231225 (+ 95us) server_negotiation.cc:407] Connection header received
0812 03:57:29.231369 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:29.231370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:29.231381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:29.231458 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:29.231623 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.232125 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.233710 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.234292 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.234369 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.234380 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:29.234640 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 03:57:29.234705 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:57:29.234882 1078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5416029670c4ee28397d5d88f4d20cc" candidate_uuid: "1d60300530c9415999215aceb165d23a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a3917aae0744673980c4d3a5aec5fc3" is_pre_election: true
I20260812 03:57:29.234998 1078 raft_consensus.cc:2454] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d60300530c9415999215aceb165d23a in term 0.
I20260812 03:57:29.235062 1366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.230984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45371 (local address 127.31.43.195:54169)
0812 03:57:29.231130 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:29.231143 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:29.231227 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:29.231463 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:29.231463 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:29.231466 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:29.231549 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:29.231551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.232229 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.232230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:29.233607 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:29.233610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.234418 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.234419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:29.234551 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.234556 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:29.234903 (+ 347us) client_negotiation.cc:240] Negotiation successful
0812 03:57:29.234963 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":69,"thread_start_us":100,"threads_started":1}
I20260812 03:57:29.235316 916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5416029670c4ee28397d5d88f4d20cc" candidate_uuid: "1d60300530c9415999215aceb165d23a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e76980dfaa4449748494e7f435770779" is_pre_election: true
I20260812 03:57:29.235419 916 raft_consensus.cc:2454] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d60300530c9415999215aceb165d23a in term 0.
I20260812 03:57:29.235462 1162 leader_election.cc:302] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d60300530c9415999215aceb165d23a, 6a3917aae0744673980c4d3a5aec5fc3; no voters:
I20260812 03:57:29.235605 1328 raft_consensus.cc:2781] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:29.235671 1328 raft_consensus.cc:482] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:29.235702 1328 raft_consensus.cc:3037] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:29.236548 1328 raft_consensus.cc:504] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:29.236708 1328 leader_election.cc:288] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [CANDIDATE]: Term 1 election: Requested vote from peers e76980dfaa4449748494e7f435770779 (127.31.43.193:45371), 6a3917aae0744673980c4d3a5aec5fc3 (127.31.43.194:45287)
I20260812 03:57:29.236888 916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5416029670c4ee28397d5d88f4d20cc" candidate_uuid: "1d60300530c9415999215aceb165d23a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e76980dfaa4449748494e7f435770779"
I20260812 03:57:29.236888 1078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5416029670c4ee28397d5d88f4d20cc" candidate_uuid: "1d60300530c9415999215aceb165d23a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a3917aae0744673980c4d3a5aec5fc3"
I20260812 03:57:29.236969 1078 raft_consensus.cc:3037] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:29.236969 916 raft_consensus.cc:3037] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:29.237739 1078 raft_consensus.cc:2454] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d60300530c9415999215aceb165d23a in term 1.
I20260812 03:57:29.237766 916 raft_consensus.cc:2454] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d60300530c9415999215aceb165d23a in term 1.
I20260812 03:57:29.237913 1162 leader_election.cc:302] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d60300530c9415999215aceb165d23a, 6a3917aae0744673980c4d3a5aec5fc3; no voters:
I20260812 03:57:29.238055 1328 raft_consensus.cc:2781] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:29.238215 1328 raft_consensus.cc:686] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [term 1 LEADER]: Becoming Leader. State: Replica: 1d60300530c9415999215aceb165d23a, State: Running, Role: LEADER
I20260812 03:57:29.238303 1328 consensus_queue.cc:237] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } }
I20260812 03:57:29.238747 1369 tablet_replica.cc:439] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: TxnStatusTablet state changed. Reason: New leader 1d60300530c9415999215aceb165d23a. Latest consensus state: current_term: 1 leader_uuid: "1d60300530c9415999215aceb165d23a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } }
I20260812 03:57:29.238853 1369 tablet_replica.cc:442] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:57:29.239163 1372 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:57:29.239259 1372 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:57:29.239243 734 catalog_manager.cc:5495] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d60300530c9415999215aceb165d23a (127.31.43.195). New cstate: current_term: 1 leader_uuid: "1d60300530c9415999215aceb165d23a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } health_report { overall_health: HEALTHY } } }
I20260812 03:57:29.287029 1362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.283717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:44557 (local address 127.0.0.1:35868)
0812 03:57:29.283807 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:29.283822 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:29.283882 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:29.284095 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:29.284096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:29.284098 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:29.284163 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:29.284166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.284919 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.284920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:29.285849 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:29.285852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.286344 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.286345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:29.286431 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.286457 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:29.286805 (+ 348us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:29.286818 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 03:57:29.286876 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 03:57:29.286936 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 03:57:29.287058 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.283775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35868 (local address 127.31.43.195:44557)
0812 03:57:29.283847 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:29.283849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:29.283861 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:57:29.283989 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:29.283990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:29.284013 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:29.284078 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:29.284263 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.284836 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.286034 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.286250 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.286321 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.286565 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:29.286743 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:29.286816 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:29.286881 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 03:57:29.286902 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:57:29.289125 916 raft_consensus.cc:1261] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779 [term 1 FOLLOWER]: Refusing update from remote peer 1d60300530c9415999215aceb165d23a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:29.289217 1078 raft_consensus.cc:1261] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3 [term 1 FOLLOWER]: Refusing update from remote peer 1d60300530c9415999215aceb165d23a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:29.289451 1328 consensus_queue.cc:1035] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:29.289588 1369 consensus_queue.cc:1035] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:29.290190 1325 tablet_replica.cc:439] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779: TxnStatusTablet state changed. Reason: New leader 1d60300530c9415999215aceb165d23a. Latest consensus state: current_term: 1 leader_uuid: "1d60300530c9415999215aceb165d23a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } }
I20260812 03:57:29.290283 1325 tablet_replica.cc:442] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:29.290225 1331 tablet_replica.cc:439] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3: TxnStatusTablet state changed. Reason: New leader 1d60300530c9415999215aceb165d23a. Latest consensus state: current_term: 1 leader_uuid: "1d60300530c9415999215aceb165d23a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } }
I20260812 03:57:29.290318 1331 tablet_replica.cc:442] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:29.291035 1369 tablet_replica.cc:439] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d60300530c9415999215aceb165d23a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } }
I20260812 03:57:29.291131 1369 tablet_replica.cc:442] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:57:29.291163 1325 tablet_replica.cc:439] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d60300530c9415999215aceb165d23a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } }
I20260812 03:57:29.291244 1325 tablet_replica.cc:442] T c5416029670c4ee28397d5d88f4d20cc P e76980dfaa4449748494e7f435770779: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:29.292569 1369 tablet_replica.cc:439] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d60300530c9415999215aceb165d23a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } }
I20260812 03:57:29.292657 1369 tablet_replica.cc:442] T c5416029670c4ee28397d5d88f4d20cc P 1d60300530c9415999215aceb165d23a: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:57:29.292868 1331 tablet_replica.cc:439] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d60300530c9415999215aceb165d23a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e76980dfaa4449748494e7f435770779" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45371 } } peers { permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 } } peers { permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 } } }
I20260812 03:57:29.292968 1331 tablet_replica.cc:442] T c5416029670c4ee28397d5d88f4d20cc P 6a3917aae0744673980c4d3a5aec5fc3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:29.295171 1325 consensus_queue.cc:1035] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a3917aae0744673980c4d3a5aec5fc3" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 45287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:29.297308 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.294234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57298 (local address 127.31.43.193:45371)
0812 03:57:29.294308 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:29.294310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:29.294320 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:29.294362 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:29.294363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:29.294389 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:29.294508 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:29.294646 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.295434 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.296164 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.296480 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.296554 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.296773 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:29.296912 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:29.296980 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:29.297179 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 03:57:29.297205 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:57:29.297935 1354 consensus_queue.cc:1035] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d60300530c9415999215aceb165d23a" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:29.304463 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.301340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:44557 (local address 127.31.43.193:48157)
0812 03:57:29.301662 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:29.301674 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:29.301747 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:29.301882 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:29.301883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:29.301884 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:29.301951 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:29.301952 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.302795 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.302796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:29.303440 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:29.303442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.303941 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.303943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:29.304024 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.304043 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:29.304272 (+ 229us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:29.304274 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:57:29.304325 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:57:29.304368 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":250,"threads_started":1}
I20260812 03:57:29.304479 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.301358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:48157 (local address 127.31.43.195:44557)
0812 03:57:29.301435 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:29.301455 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:29.301748 (+ 293us) server_negotiation.cc:407] Connection header received
0812 03:57:29.301800 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:29.301801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:29.301811 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:29.301880 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:29.302002 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.302708 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.303534 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.303827 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.303903 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.304143 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:29.304179 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:29.304267 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:29.304329 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 03:57:29.304368 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:57:29.316340 1390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.313027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45371 (local address 127.31.43.195:37463)
0812 03:57:29.313305 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:29.313317 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:29.313414 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:29.313554 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:29.313555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:29.313561 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:29.313627 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:29.313629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.314389 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.314390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:29.315474 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:29.315478 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.315859 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.315861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:29.315918 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.315948 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:29.316152 (+ 204us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:29.316154 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:57:29.316198 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 03:57:29.316248 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":203,"threads_started":1}
I20260812 03:57:29.316332 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.313135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:37463 (local address 127.31.43.193:45371)
0812 03:57:29.313204 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:29.313206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:29.313393 (+ 187us) server_negotiation.cc:407] Connection header received
0812 03:57:29.313477 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:29.313478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:29.313489 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:29.313550 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:29.313705 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.314320 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.315550 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.315768 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.315844 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.316033 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:29.316070 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:29.316138 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:29.316199 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 03:57:29.316225 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:57:29.328071 734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:29.328212 734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:29.329054 734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d87eedade95248ba89fab68e0ae6a592: Sending DeleteTablet for 3 replicas of tablet d17b2c19a7854270b35d82227b553011
I20260812 03:57:29.329373 891 tablet_service.cc:1514] Processing DeleteTablet for tablet d17b2c19a7854270b35d82227b553011 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:29 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:57296
I20260812 03:57:29.329473 1346 tablet_replica.cc:330] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779: stopping tablet replica
I20260812 03:57:29.329516 1346 raft_consensus.cc:2229] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:57:29.329530 1054 tablet_service.cc:1514] Processing DeleteTablet for tablet d17b2c19a7854270b35d82227b553011 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:29 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:35974
I20260812 03:57:29.329622 1346 raft_consensus.cc:2258] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:29.329845 1213 tablet_service.cc:1514] Processing DeleteTablet for tablet d17b2c19a7854270b35d82227b553011 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:29 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:35858
I20260812 03:57:29.330118 1346 ts_tablet_manager.cc:1905] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:29.330204 1395 tablet_replica.cc:330] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3: stopping tablet replica
I20260812 03:57:29.330267 1395 raft_consensus.cc:2229] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:29.330313 1395 raft_consensus.cc:2258] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:29.330714 1395 ts_tablet_manager.cc:1905] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:29.332537 1346 ts_tablet_manager.cc:1918] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:29.332599 1346 log.cc:1196] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-0/wal/wals/d17b2c19a7854270b35d82227b553011
I20260812 03:57:29.332787 1395 ts_tablet_manager.cc:1918] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:29.332849 1395 log.cc:1196] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/d17b2c19a7854270b35d82227b553011
I20260812 03:57:29.332854 1346 ts_tablet_manager.cc:1939] T d17b2c19a7854270b35d82227b553011 P e76980dfaa4449748494e7f435770779: Deleting consensus metadata
I20260812 03:57:29.333092 1395 ts_tablet_manager.cc:1939] T d17b2c19a7854270b35d82227b553011 P 6a3917aae0744673980c4d3a5aec5fc3: Deleting consensus metadata
I20260812 03:57:29.333701 720 catalog_manager.cc:4841] TS 6a3917aae0744673980c4d3a5aec5fc3 (127.31.43.194:45287): tablet d17b2c19a7854270b35d82227b553011 (table test-table [id=cf5be45d9b414a5197d8c51c1cb620e0]) successfully deleted
I20260812 03:57:29.334334 1396 tablet_replica.cc:330] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a: stopping tablet replica
I20260812 03:57:29.334419 718 catalog_manager.cc:4841] TS e76980dfaa4449748494e7f435770779 (127.31.43.193:45371): tablet d17b2c19a7854270b35d82227b553011 (table test-table [id=cf5be45d9b414a5197d8c51c1cb620e0]) successfully deleted
I20260812 03:57:29.334482 1396 raft_consensus.cc:2229] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:29.334539 1396 raft_consensus.cc:2258] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:29.334849 1396 ts_tablet_manager.cc:1905] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:29.337582 1396 ts_tablet_manager.cc:1918] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:29.337661 1396 log.cc:1196] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/d17b2c19a7854270b35d82227b553011
I20260812 03:57:29.337903 1396 ts_tablet_manager.cc:1939] T d17b2c19a7854270b35d82227b553011 P 1d60300530c9415999215aceb165d23a: Deleting consensus metadata
I20260812 03:57:29.338353 721 catalog_manager.cc:4841] TS 1d60300530c9415999215aceb165d23a (127.31.43.195:44557): tablet d17b2c19a7854270b35d82227b553011 (table test-table [id=cf5be45d9b414a5197d8c51c1cb620e0]) successfully deleted
Aug 12 03:57:29 dist-test-slave-8tdl krb5kdc[685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507048, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.43.193@KRBTEST.COM
I20260812 03:57:29.351178 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.338987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.31.43.193:45371)
0812 03:57:29.339110 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:29.339113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:29.339330 (+ 217us) server_negotiation.cc:407] Connection header received
0812 03:57:29.339390 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:29.339392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:29.339411 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:29.339477 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:29.340288 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.341008 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.342453 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.342778 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.342829 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.347012 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:29.347025 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:29.347025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:29.347036 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:29.349313 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:29.349840 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:29.349844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:29.349845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:29.349934 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:29.350167 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:29.350168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:29.350168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:29.350374 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:29.350577 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:29.350889 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 03:57:29.350991 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 03:57:29.352448 960 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57310
I20260812 03:57:29.362596 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:29.353621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55948 (local address 127.31.43.254:40763)
0812 03:57:29.353671 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:29.353673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:29.353894 (+ 221us) server_negotiation.cc:407] Connection header received
0812 03:57:29.353944 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:29.353945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:29.353958 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:29.354015 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:29.355162 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.356027 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.357102 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:29.357431 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:29.357514 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:29.359155 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:29.359157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:29.359157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:29.359165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:29.361110 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:29.361605 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:29.361607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:29.361607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:29.361678 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:29.361911 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:29.361913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:29.361913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:29.362074 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:29.362187 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:29.362381 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 03:57:29.362484 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 03:57:29.363135 734 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55948
I20260812 03:57:29.364066 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 821
I20260812 03:57:29.373873 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 979
I20260812 03:57:29.385232 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 1142
I20260812 03:57:29.395565 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 699
2026-08-12T03:57:29Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2857 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:29 dist-test-slave-8tdl krb5kdc[1403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:29 dist-test-slave-8tdl krb5kdc[1403](info): set up 2 sockets
Aug 12 03:57:29 dist-test-slave-8tdl krb5kdc[1403](info): commencing operation
krb5kdc: starting...
W20260812 03:57:30.526077 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:57:30 dist-test-slave-8tdl krb5kdc[1403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:57:30.667469 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:37733
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:37733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:30.767308 1417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:30.767592 1417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:30.767632 1417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:30.768947 1417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:30.769025 1417 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:57:30.769049 1417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:30.769073 1417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:30.769114 1417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:30.769145 1417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:30.770968 1417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:37733
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:37733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:30.771297 1417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:30.771559 1417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:30.777711 1425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:30.777727 1422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:30.777727 1423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:30.777895 1417 server_base.cc:1034] running on GCE node
I20260812 03:57:30.778231 1417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:30.778571 1417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:30.779825 1417 hybrid_clock.cc:648] HybridClock initialized: now 1786507050779827 us; error 71 us; skew 500 ppm
Aug 12 03:57:30 dist-test-slave-8tdl krb5kdc[1403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507050, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:30.783021 1417 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:57:30.783658 1417 webserver.cc:460] Webserver started at http://127.31.43.254:45451/ using document root <none> and password file <none>
I20260812 03:57:30.783942 1417 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:30.784019 1417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:30.784185 1417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:30.785432 1417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "a8c0df41123c413eb57f172d1fceaa97"
format_stamp: "Formatted at 2026-08-12 03:57:30 on dist-test-slave-8tdl"
server_key: "e97d6fc35bd1459ffbc884cb38bc9f70"
server_key_iv: "024ecf3db10982fd18394c29e4266a92"
server_key_version: "encryptionkey@0"
I20260812 03:57:30.785856 1417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "a8c0df41123c413eb57f172d1fceaa97"
format_stamp: "Formatted at 2026-08-12 03:57:30 on dist-test-slave-8tdl"
server_key: "e97d6fc35bd1459ffbc884cb38bc9f70"
server_key_iv: "024ecf3db10982fd18394c29e4266a92"
server_key_version: "encryptionkey@0"
I20260812 03:57:30.787698 1417 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 03:57:30.788774 1433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:30.789065 1417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:30.789196 1417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "a8c0df41123c413eb57f172d1fceaa97"
format_stamp: "Formatted at 2026-08-12 03:57:30 on dist-test-slave-8tdl"
server_key: "e97d6fc35bd1459ffbc884cb38bc9f70"
server_key_iv: "024ecf3db10982fd18394c29e4266a92"
server_key_version: "encryptionkey@0"
I20260812 03:57:30.789288 1417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:30.812245 1417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:30.815018 1417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:30.815222 1417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:30.820834 1417 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:37733
I20260812 03:57:30.820890 1505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:37733 every 8 connection(s)
I20260812 03:57:30.821267 1417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:30.821975 1506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:30.824324 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 1417
I20260812 03:57:30.824447 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:30.824626 31919 external_mini_cluster.cc:1428] Setting key c35745e971fb6fb5d1e2aee11296b55a
I20260812 03:57:30.824625 1506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97: Bootstrap starting.
I20260812 03:57:30.825546 1506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:30.825975 1506 log.cc:824] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:30.826861 1506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97: No bootstrap required, opened a new log
I20260812 03:57:30.829389 1506 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c0df41123c413eb57f172d1fceaa97" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37733 } }
I20260812 03:57:30.829572 1506 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:30.829598 1506 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c0df41123c413eb57f172d1fceaa97, State: Initialized, Role: FOLLOWER
I20260812 03:57:30.829747 1506 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c0df41123c413eb57f172d1fceaa97" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37733 } }
I20260812 03:57:30.829834 1506 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:30.829875 1506 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:30.829928 1506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:30.830555 1506 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c0df41123c413eb57f172d1fceaa97" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37733 } }
I20260812 03:57:30.830708 1506 leader_election.cc:302] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8c0df41123c413eb57f172d1fceaa97; no voters:
I20260812 03:57:30.830930 1506 leader_election.cc:288] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:30.830979 1512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:30.831173 1512 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [term 1 LEADER]: Becoming Leader. State: Replica: a8c0df41123c413eb57f172d1fceaa97, State: Running, Role: LEADER
I20260812 03:57:30.831240 1506 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:30.831269 1512 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c0df41123c413eb57f172d1fceaa97" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37733 } }
I20260812 03:57:30.831683 1513 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8c0df41123c413eb57f172d1fceaa97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c0df41123c413eb57f172d1fceaa97" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37733 } } }
I20260812 03:57:30.831696 1514 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8c0df41123c413eb57f172d1fceaa97. Latest consensus state: current_term: 1 leader_uuid: "a8c0df41123c413eb57f172d1fceaa97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c0df41123c413eb57f172d1fceaa97" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37733 } } }
I20260812 03:57:30.832015 1514 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:30.832267 1522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:30.831903 1513 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:30.832901 1522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:57:30 dist-test-slave-8tdl krb5kdc[1403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:30.834859 1522 catalog_manager.cc:1347] Generated new cluster ID: 42a5c6ab318d4e19aa428ce827b1137a
I20260812 03:57:30.834931 1522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:30.842866 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:30.825317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33896 (local address 127.31.43.254:37733)
0812 03:57:30.825643 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:30.825654 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:30.825693 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:57:30.826345 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:30.826353 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:30.826436 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:30.826615 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:30.827398 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:30.828893 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:30.829825 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:30.830360 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:30.830518 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:30.834604 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:30.834608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:30.834609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:30.834623 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:30.840809 (+ 6186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:30.841353 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:30.841357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:30.841357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:30.841503 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:30.841732 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:30.841733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:30.841733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:30.842109 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:30.842281 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:30.842406 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 03:57:30.842574 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":100,"threads_started":1}
I20260812 03:57:30.859237 1522 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:30.859862 1522 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:30.870734 1522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97: Generated new TSK 0
I20260812 03:57:30.871013 1522 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 03:57:30.907052 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:37733
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:30.996987 1538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:30.997229 1538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:30.997259 1538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:30.998420 1538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:30.998459 1538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:30.998481 1538 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:57:30.998493 1538 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:57:30.998510 1538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:31.000124 1538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.43.254:37733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:31.000355 1538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:31.000599 1538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:31.006237 1546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:31.006238 1547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:31.006238 1549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:31.006508 1538 server_base.cc:1034] running on GCE node
I20260812 03:57:31.006824 1538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:31.007073 1538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:31.008316 1538 hybrid_clock.cc:648] HybridClock initialized: now 1786507051008302 us; error 43 us; skew 500 ppm
Aug 12 03:57:31 dist-test-slave-8tdl krb5kdc[1403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507051, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:31.011528 1538 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 03:57:31.012152 1538 webserver.cc:460] Webserver started at http://127.31.43.193:44111/ using document root <none> and password file <none>
I20260812 03:57:31.012415 1538 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:31.012488 1538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:31.012650 1538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:31.013832 1538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "842f35c85ebb41dda94f9909e89de3a4"
format_stamp: "Formatted at 2026-08-12 03:57:31 on dist-test-slave-8tdl"
server_key: "d5ca0b8e1f048d0526cd604f32c83abf"
server_key_iv: "0ded378a0c60dad45b84a383f2071da2"
server_key_version: "encryptionkey@0"
I20260812 03:57:31.014248 1538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "842f35c85ebb41dda94f9909e89de3a4"
format_stamp: "Formatted at 2026-08-12 03:57:31 on dist-test-slave-8tdl"
server_key: "d5ca0b8e1f048d0526cd604f32c83abf"
server_key_iv: "0ded378a0c60dad45b84a383f2071da2"
server_key_version: "encryptionkey@0"
I20260812 03:57:31.015900 1538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:57:31.016978 1558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:31.017288 1538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:31.017437 1538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "842f35c85ebb41dda94f9909e89de3a4"
format_stamp: "Formatted at 2026-08-12 03:57:31 on dist-test-slave-8tdl"
server_key: "d5ca0b8e1f048d0526cd604f32c83abf"
server_key_iv: "0ded378a0c60dad45b84a383f2071da2"
server_key_version: "encryptionkey@0"
I20260812 03:57:31.017534 1538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:31.035784 1538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:31.038383 1538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:31.038571 1538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:31.039294 1538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:31.039358 1538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:31.039395 1538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:31.039422 1538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:57:31 dist-test-slave-8tdl krb5kdc[1403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507051, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:31.047492 1538 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:35569
I20260812 03:57:31.047909 1538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:57:31.047902 1696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:35569 every 8 connection(s)
I20260812 03:57:31.050889 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.039951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:47911 (local address 127.31.43.254:37733)
0812 03:57:31.040094 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.040097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.040257 (+ 160us) server_negotiation.cc:407] Connection header received
0812 03:57:31.040909 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.040911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.040928 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.041005 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:31.041759 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.042632 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.043987 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.044225 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.044318 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.047201 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:31.047203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:31.047203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:31.047212 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:31.048999 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.049472 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.049473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.049474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.049549 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.049959 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.049961 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.049961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.050095 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:31.050205 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.050622 (+ 417us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.050776 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260812 03:57:31.051025 1569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.039883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37733 (local address 127.31.43.193:47911)
0812 03:57:31.040099 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.040153 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.040749 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.041117 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.041121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.041397 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:31.041641 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.041646 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.042776 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.042777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.043834 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.043838 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.044343 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.044344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.044513 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.045152 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:31.045164 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:31.047078 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:31.049102 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.049110 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.049112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.049359 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.049673 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.049674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.049674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.049795 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.050190 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:31.050196 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:31.050414 (+ 218us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.050565 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.050782 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
I20260812 03:57:31.051658 1697 heartbeater.cc:347] Connected to a master server at 127.31.43.254:37733
I20260812 03:57:31.051755 1697 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:31.051957 1697 heartbeater.cc:511] Master 127.31.43.254:37733 requested a full tablet report, sending...
I20260812 03:57:31.052608 1455 ts_manager.cc:194] Registered new tserver with Master: 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193:35569)
I20260812 03:57:31.053412 1455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:47911
I20260812 03:57:31.054051 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 1538
I20260812 03:57:31.054169 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:57:31.054407 31919 external_mini_cluster.cc:1428] Setting key ffe021a4352ea72f0ce74a6518e21095
I20260812 03:57:31.060688 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.053002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:45407 (local address 127.31.43.254:37733)
0812 03:57:31.053631 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.053633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.053641 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:57:31.053667 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.053667 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.053692 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.053756 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:31.054456 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.055263 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.056359 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.056605 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.056660 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.057982 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:31.057984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:31.057984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:31.057992 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:31.059429 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.059802 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.059803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.059804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.059868 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.060102 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.060103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.060104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.060235 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:31.060313 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.060485 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.060582 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609}
I20260812 03:57:31.060693 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.052935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37733 (local address 127.31.43.193:45407)
0812 03:57:31.053093 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.053104 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.053152 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.053786 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.053787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.054237 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:31.054330 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.054333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.055387 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.055388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.056265 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.056269 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.056683 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.056685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.056782 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.057389 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:31.057395 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:31.057829 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:31.059525 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.059526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.059527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.059719 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.059950 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.059951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.059951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.060027 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.060304 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:31.060305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:31.060360 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.060457 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.060585 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 03:57:31.086548 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:37733
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:31.176746 1709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:31.176990 1709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:31.177019 1709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:31.178198 1709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:31.178237 1709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:31.178259 1709 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:57:31.178272 1709 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:57:31.178288 1709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:31.179855 1709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.43.254:37733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:31.180105 1709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:31.180306 1709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:31.185891 1714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:31.185920 1715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:31.186160 1709 server_base.cc:1034] running on GCE node
W20260812 03:57:31.185905 1717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:31.186503 1709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:31.186861 1709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:31.188107 1709 hybrid_clock.cc:648] HybridClock initialized: now 1786507051188097 us; error 40 us; skew 500 ppm
Aug 12 03:57:31 dist-test-slave-8tdl krb5kdc[1403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507051, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:31.190909 1709 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 03:57:31.191515 1709 webserver.cc:460] Webserver started at http://127.31.43.194:45559/ using document root <none> and password file <none>
I20260812 03:57:31.191809 1709 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:31.191890 1709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:31.192049 1709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:31.193480 1709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "b09f63f9ed1349dfa00cfa58e68377c5"
format_stamp: "Formatted at 2026-08-12 03:57:31 on dist-test-slave-8tdl"
server_key: "f721aa2dc996601050266fb5b51a9809"
server_key_iv: "b1995b79c46ed734fba788fe2a3ee9cd"
server_key_version: "encryptionkey@0"
I20260812 03:57:31.193902 1709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "b09f63f9ed1349dfa00cfa58e68377c5"
format_stamp: "Formatted at 2026-08-12 03:57:31 on dist-test-slave-8tdl"
server_key: "f721aa2dc996601050266fb5b51a9809"
server_key_iv: "b1995b79c46ed734fba788fe2a3ee9cd"
server_key_version: "encryptionkey@0"
I20260812 03:57:31.195604 1709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 03:57:31.196537 1725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:31.196769 1709 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:31.196895 1709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "b09f63f9ed1349dfa00cfa58e68377c5"
format_stamp: "Formatted at 2026-08-12 03:57:31 on dist-test-slave-8tdl"
server_key: "f721aa2dc996601050266fb5b51a9809"
server_key_iv: "b1995b79c46ed734fba788fe2a3ee9cd"
server_key_version: "encryptionkey@0"
I20260812 03:57:31.196983 1709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:31.204613 1709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:31.206851 1709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:31.207033 1709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:31.207610 1709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:31.207675 1709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:31.207711 1709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:31.207738 1709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:57:31 dist-test-slave-8tdl krb5kdc[1403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507051, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:31.215749 1709 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:35067
I20260812 03:57:31.216153 1709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:57:31.216315 1862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:35067 every 8 connection(s)
I20260812 03:57:31.219424 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.208119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:35357 (local address 127.31.43.254:37733)
0812 03:57:31.208199 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.208202 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.208476 (+ 274us) server_negotiation.cc:407] Connection header received
0812 03:57:31.209304 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.209306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.209321 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.209383 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:31.210386 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.211053 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.212819 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.213176 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.213258 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.216236 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:31.216238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:31.216238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:31.216248 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:31.217880 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.218338 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.218339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.218340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.218404 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.218703 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.218704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.218705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.218839 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:31.218922 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.219222 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.219316 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:57:31.219686 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.208095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37733 (local address 127.31.43.194:35357)
0812 03:57:31.208314 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.208374 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.209162 (+ 788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.209475 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.209481 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.209935 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:31.210270 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.210278 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.211201 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.211203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.212717 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.212721 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.213306 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.213307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.213501 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.214133 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:31.214145 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:31.216121 (+ 1976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:31.217992 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.218000 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.218003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.218249 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.218480 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.218480 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.218481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.218615 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.218912 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:31.218932 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:31.219082 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.219200 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.219404 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260812 03:57:31.220317 1864 heartbeater.cc:347] Connected to a master server at 127.31.43.254:37733
I20260812 03:57:31.220444 1864 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:31.220652 1864 heartbeater.cc:511] Master 127.31.43.254:37733 requested a full tablet report, sending...
I20260812 03:57:31.221135 1454 ts_manager.cc:194] Registered new tserver with Master: b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194:35067)
I20260812 03:57:31.221604 1454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:35357
I20260812 03:57:31.223222 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 1709
I20260812 03:57:31.223327 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:57:31.223515 31919 external_mini_cluster.cc:1428] Setting key dd0b8007e3bc4a3a7a0c459f9f30b223
I20260812 03:57:31.228785 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.221943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37733 (local address 127.31.43.194:52223)
0812 03:57:31.222117 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.222130 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.222194 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.222336 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.222337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.222552 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:31.222616 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.222618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.223529 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.223530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.224375 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.224378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.224895 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.224896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.224958 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.225476 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:31.225482 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:31.225804 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:31.227599 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.227600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.227601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.227841 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.228097 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.228098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.228098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.228153 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.228523 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:31.228524 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:31.228564 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.228619 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.228710 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":94,"threads_started":1}
I20260812 03:57:31.228906 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.222025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:52223 (local address 127.31.43.254:37733)
0812 03:57:31.222071 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.222073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.222198 (+ 125us) server_negotiation.cc:407] Connection header received
0812 03:57:31.222253 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.222254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.222263 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.222313 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:31.222728 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.223433 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.224500 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.224812 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.224891 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.225892 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:31.225894 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:31.225894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:31.225902 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:31.227506 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.227942 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.227943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.227943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.228008 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.228218 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.228219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.228219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.228455 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:31.228531 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.228670 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.228771 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 03:57:31.257944 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:37733
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:31.349051 1874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:31.349344 1874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:31.349393 1874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:31.350771 1874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:31.350840 1874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:31.350876 1874 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:57:31.350907 1874 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:57:31.350945 1874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:31.352653 1874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.43.254:37733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:31.352962 1874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:31.353220 1874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:31.358836 1880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:31.358801 1879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:31.358906 1874 server_base.cc:1034] running on GCE node
W20260812 03:57:31.358824 1882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:31.359330 1874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:31.359623 1874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:31.360828 1874 hybrid_clock.cc:648] HybridClock initialized: now 1786507051360808 us; error 41 us; skew 500 ppm
Aug 12 03:57:31 dist-test-slave-8tdl krb5kdc[1403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507051, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:31.363832 1874 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 03:57:31.364471 1874 webserver.cc:460] Webserver started at http://127.31.43.195:34129/ using document root <none> and password file <none>
I20260812 03:57:31.364825 1874 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:31.364913 1874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:31.365072 1874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:31.366288 1874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "a2a3fd59e9a54babb012b2f53e1da667"
format_stamp: "Formatted at 2026-08-12 03:57:31 on dist-test-slave-8tdl"
server_key: "9dde40e5dc7f96f4e1fab48794a8f9fb"
server_key_iv: "c5e1c6f1fa4dfaa9aee8c88348db5e49"
server_key_version: "encryptionkey@0"
I20260812 03:57:31.366765 1874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "a2a3fd59e9a54babb012b2f53e1da667"
format_stamp: "Formatted at 2026-08-12 03:57:31 on dist-test-slave-8tdl"
server_key: "9dde40e5dc7f96f4e1fab48794a8f9fb"
server_key_iv: "c5e1c6f1fa4dfaa9aee8c88348db5e49"
server_key_version: "encryptionkey@0"
I20260812 03:57:31.368302 1874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:31.369160 1889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:31.369367 1874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:31.369485 1874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "a2a3fd59e9a54babb012b2f53e1da667"
format_stamp: "Formatted at 2026-08-12 03:57:31 on dist-test-slave-8tdl"
server_key: "9dde40e5dc7f96f4e1fab48794a8f9fb"
server_key_iv: "c5e1c6f1fa4dfaa9aee8c88348db5e49"
server_key_version: "encryptionkey@0"
I20260812 03:57:31.369575 1874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:31.386356 1874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:31.389055 1874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:31.389253 1874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:31.389905 1874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:31.389971 1874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:31.390026 1874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:31.390048 1874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:57:31 dist-test-slave-8tdl krb5kdc[1403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507051, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:31.398070 1874 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:42951
I20260812 03:57:31.398169 2023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:42951 every 8 connection(s)
I20260812 03:57:31.398556 1874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:57:31.400951 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.390463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:58087 (local address 127.31.43.254:37733)
0812 03:57:31.390517 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.390524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.390805 (+ 281us) server_negotiation.cc:407] Connection header received
0812 03:57:31.391401 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.391403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.391418 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.391468 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:31.392416 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.393202 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.394325 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.394576 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.394645 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.397635 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:31.397637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:31.397637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:31.397647 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:31.399477 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.399956 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.399992 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.399993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.400062 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.400367 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.400368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.400369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.400500 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:31.400554 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.400813 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.400884 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:57:31.401197 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.390419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37733 (local address 127.31.43.195:58087)
0812 03:57:31.390626 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.390698 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.391283 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.391572 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.391576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.391977 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:31.392295 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.392301 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.393318 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.393319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.394210 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.394213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.394661 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.394663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.394882 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.395569 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:31.395581 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:31.397505 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:31.399583 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.399590 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.399592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.399863 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.400154 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.400156 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.400156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.400273 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.400563 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:31.400568 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:31.400697 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.400785 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.400929 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
I20260812 03:57:31.401813 2024 heartbeater.cc:347] Connected to a master server at 127.31.43.254:37733
I20260812 03:57:31.401902 2024 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:31.402154 2024 heartbeater.cc:511] Master 127.31.43.254:37733 requested a full tablet report, sending...
I20260812 03:57:31.402712 1455 ts_manager.cc:194] Registered new tserver with Master: a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951)
I20260812 03:57:31.403200 1455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:58087
I20260812 03:57:31.404985 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 1874
I20260812 03:57:31.405097 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:57:31.405289 31919 external_mini_cluster.cc:1428] Setting key b7f46acff655bcdecbd09eadbe82d3d1
I20260812 03:57:31.406329 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:57:31.410390 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.403314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37733 (local address 127.31.43.195:47021)
0812 03:57:31.403491 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.403503 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.403578 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.403710 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.403711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.404014 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:31.404090 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.404093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.404783 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.404784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.405604 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.405606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.406015 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.406015 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.406230 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.406758 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:31.406764 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:31.407119 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:31.409063 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.409064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.409065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.409338 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.409537 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.409538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.409538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.409627 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.409873 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:31.409874 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:31.410040 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.410149 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.410287 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260812 03:57:31.410429 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.403450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:47021 (local address 127.31.43.254:37733)
0812 03:57:31.403501 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.403503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.403587 (+ 84us) server_negotiation.cc:407] Connection header received
0812 03:57:31.403647 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.403648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.403656 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.403701 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:31.404179 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.404658 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.405704 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.405918 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.405983 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.407232 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:31.407234 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:31.407234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:31.407243 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:31.408965 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.409418 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.409419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.409419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.409474 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.409698 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.409698 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.409699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.409807 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:31.409860 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.410219 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.410319 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 03:57:31 dist-test-slave-8tdl krb5kdc[1403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:57:31 dist-test-slave-8tdl krb5kdc[1403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:31.435416 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.427357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33904 (local address 127.31.43.254:37733)
0812 03:57:31.427433 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.427435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.427445 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:31.427478 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.427479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.427489 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.427555 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:31.428178 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.428698 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.429495 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.429698 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.429777 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.432720 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:31.432722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:31.432723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:31.432732 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:31.434132 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.434505 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.434507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.434508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.434569 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.434820 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.434821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.434822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.434958 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:31.435068 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.435205 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.435299 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:57:31.437264 1455 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:31.439059 1455 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:31.446656 2047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.441874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:35067 (local address 127.0.0.1:38492)
0812 03:57:31.442088 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.442134 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.442209 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.443195 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.443196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.443198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:31.443246 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.443248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.444473 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.444474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.445451 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.445454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.446345 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.446346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.446417 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.446431 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.446497 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.446545 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":83,"threads_started":1}
I20260812 03:57:31.446787 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.442190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.31.43.194:35067)
0812 03:57:31.442814 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.442818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.442828 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:31.442877 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.442879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.442894 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.443129 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:31.443323 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.444388 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.445552 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.446252 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.446334 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.446401 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.446535 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.446587 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":77,"threads_started":1}
I20260812 03:57:31.447178 2045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.441872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:42951 (local address 127.0.0.1:53466)
0812 03:57:31.442085 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.442132 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.442209 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.442787 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.442791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.442793 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:31.442884 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.442886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.444346 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.444347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.445742 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.445745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.446747 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.446749 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.446978 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.446982 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.447034 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.447094 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":51,"spinlock_wait_cycles":768,"thread_start_us":127,"threads_started":1}
I20260812 03:57:31.447477 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.442371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53466 (local address 127.31.43.195:42951)
0812 03:57:31.442569 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.442574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.442586 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:57:31.442629 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.442631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.442645 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.442765 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:31.443048 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.444263 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.445843 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.446589 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.446668 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.446752 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.447303 (+ 551us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.447363 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":83,"threads_started":1}
I20260812 03:57:31.447594 1783 tablet_service.cc:1467] Processing CreateTablet for tablet b35a72c281ef4ead9c6e1ebf89bdffd3 (DEFAULT_TABLE table=test-table [id=74aad9e1758b4abda99fcbc6f0e42649]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:31.448024 1783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b35a72c281ef4ead9c6e1ebf89bdffd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:31.448240 1948 tablet_service.cc:1467] Processing CreateTablet for tablet b35a72c281ef4ead9c6e1ebf89bdffd3 (DEFAULT_TABLE table=test-table [id=74aad9e1758b4abda99fcbc6f0e42649]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:31.448589 1948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b35a72c281ef4ead9c6e1ebf89bdffd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:31.451395 2055 tablet_bootstrap.cc:492] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667: Bootstrap starting.
I20260812 03:57:31.451663 2054 tablet_bootstrap.cc:492] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5: Bootstrap starting.
I20260812 03:57:31.452507 2054 tablet_bootstrap.cc:654] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:31.452605 2055 tablet_bootstrap.cc:654] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:31.452639 2046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.441875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:35569 (local address 127.0.0.1:48144)
0812 03:57:31.442095 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.442135 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.442209 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.443269 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.443270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.443272 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:31.443326 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.443327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.444251 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.444252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.445742 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.445745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.452201 (+ 6456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.452203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.452299 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.452307 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.452376 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.452443 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":90,"thread_start_us":133,"threads_started":1}
I20260812 03:57:31.452991 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.442555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48144 (local address 127.31.43.193:35569)
0812 03:57:31.442925 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.442930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.442940 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:31.443015 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.443017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.443038 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.443260 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:31.443406 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.444151 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.446799 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.452087 (+ 5288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.452177 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.452244 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.452808 (+ 564us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.452868 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":112,"threads_started":1}
I20260812 03:57:31.452982 2054 log.cc:824] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:31.452991 2055 log.cc:824] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:31.453769 1615 tablet_service.cc:1467] Processing CreateTablet for tablet b35a72c281ef4ead9c6e1ebf89bdffd3 (DEFAULT_TABLE table=test-table [id=74aad9e1758b4abda99fcbc6f0e42649]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:31.453924 2055 tablet_bootstrap.cc:492] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667: No bootstrap required, opened a new log
I20260812 03:57:31.453996 2055 ts_tablet_manager.cc:1397] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:57:31.454195 1615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b35a72c281ef4ead9c6e1ebf89bdffd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:31.455245 2054 tablet_bootstrap.cc:492] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5: No bootstrap required, opened a new log
I20260812 03:57:31.455338 2054 ts_tablet_manager.cc:1397] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 03:57:31.456182 2055 raft_consensus.cc:348] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } }
I20260812 03:57:31.456372 2055 raft_consensus.cc:374] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:31.456410 2055 raft_consensus.cc:729] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a3fd59e9a54babb012b2f53e1da667, State: Initialized, Role: FOLLOWER
I20260812 03:57:31.456542 2055 consensus_queue.cc:260] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } }
I20260812 03:57:31.456985 2055 ts_tablet_manager.cc:1428] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:57:31.457034 2024 heartbeater.cc:503] Master 127.31.43.254:37733 was elected leader, sending a full tablet report...
I20260812 03:57:31.457185 2054 raft_consensus.cc:348] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } }
I20260812 03:57:31.457345 2054 raft_consensus.cc:374] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:31.457386 2054 raft_consensus.cc:729] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b09f63f9ed1349dfa00cfa58e68377c5, State: Initialized, Role: FOLLOWER
I20260812 03:57:31.457520 2054 consensus_queue.cc:260] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } }
I20260812 03:57:31.457819 2054 ts_tablet_manager.cc:1428] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:57:31.457935 1864 heartbeater.cc:503] Master 127.31.43.254:37733 was elected leader, sending a full tablet report...
I20260812 03:57:31.458277 2061 tablet_bootstrap.cc:492] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4: Bootstrap starting.
I20260812 03:57:31.459056 2061 tablet_bootstrap.cc:654] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:31.459380 2061 log.cc:824] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:31.460166 2061 tablet_bootstrap.cc:492] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4: No bootstrap required, opened a new log
I20260812 03:57:31.460230 2061 ts_tablet_manager.cc:1397] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:57:31.461699 2061 raft_consensus.cc:348] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } }
I20260812 03:57:31.461843 2061 raft_consensus.cc:374] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:31.461866 2061 raft_consensus.cc:729] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842f35c85ebb41dda94f9909e89de3a4, State: Initialized, Role: FOLLOWER
I20260812 03:57:31.462038 2061 consensus_queue.cc:260] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } }
I20260812 03:57:31.462354 2061 ts_tablet_manager.cc:1428] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:57:31.462431 1697 heartbeater.cc:503] Master 127.31.43.254:37733 was elected leader, sending a full tablet report...
W20260812 03:57:31.467224 1865 tablet.cc:2369] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:57:31.548985 1698 tablet.cc:2369] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:57:31.568265 2064 raft_consensus.cc:482] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:31.568426 2064 raft_consensus.cc:504] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } }
I20260812 03:57:31.568776 2064 leader_election.cc:288] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951), b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194:35067)
I20260812 03:57:31.572124 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.569225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:48465 (local address 127.31.43.195:42951)
0812 03:57:31.569296 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.569298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.569403 (+ 105us) server_negotiation.cc:407] Connection header received
0812 03:57:31.569480 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.569481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.569490 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.569565 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:31.569748 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.570220 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.571360 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.571743 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.571842 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.571851 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.572004 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.572041 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:57:31.572124 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.569124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:44051 (local address 127.31.43.194:35067)
0812 03:57:31.569178 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.569180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.569296 (+ 116us) server_negotiation.cc:407] Connection header received
0812 03:57:31.569406 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.569407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.569417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.569503 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:31.569658 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.570191 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.571357 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.571737 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.571829 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.571840 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.571997 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.572037 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:57:31.572150 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.569004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:35067 (local address 127.31.43.193:44051)
0812 03:57:31.569201 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.569213 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.569319 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.569512 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.569513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.569515 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:31.569577 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.569579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.570296 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.570297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.571261 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.571264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.571829 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.571830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.571913 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.571917 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.571986 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.572037 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":35,"thread_start_us":104,"threads_started":1}
I20260812 03:57:31.572152 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.569012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:42951 (local address 127.31.43.193:48465)
0812 03:57:31.569323 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.569332 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.569421 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.569587 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.569588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.569591 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:31.569667 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.569669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.570308 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.570309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.571255 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.571258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.571842 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.571843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.571927 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.571930 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.571999 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.572044 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":32,"thread_start_us":102,"threads_started":1}
I20260812 03:57:31.572574 1807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b35a72c281ef4ead9c6e1ebf89bdffd3" candidate_uuid: "842f35c85ebb41dda94f9909e89de3a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" is_pre_election: true
I20260812 03:57:31.572808 1807 raft_consensus.cc:2454] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842f35c85ebb41dda94f9909e89de3a4 in term 0.
I20260812 03:57:31.573086 1562 leader_election.cc:302] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842f35c85ebb41dda94f9909e89de3a4, b09f63f9ed1349dfa00cfa58e68377c5; no voters:
I20260812 03:57:31.573067 1974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b35a72c281ef4ead9c6e1ebf89bdffd3" candidate_uuid: "842f35c85ebb41dda94f9909e89de3a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a3fd59e9a54babb012b2f53e1da667" is_pre_election: true
I20260812 03:57:31.573228 1974 raft_consensus.cc:2454] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842f35c85ebb41dda94f9909e89de3a4 in term 0.
I20260812 03:57:31.573244 2064 raft_consensus.cc:2781] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:31.573295 2064 raft_consensus.cc:482] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:31.573333 2064 raft_consensus.cc:3037] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:31.574193 2064 raft_consensus.cc:504] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } }
I20260812 03:57:31.574362 2064 leader_election.cc:288] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [CANDIDATE]: Term 1 election: Requested vote from peers a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951), b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194:35067)
I20260812 03:57:31.574535 1974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b35a72c281ef4ead9c6e1ebf89bdffd3" candidate_uuid: "842f35c85ebb41dda94f9909e89de3a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a3fd59e9a54babb012b2f53e1da667"
I20260812 03:57:31.574563 1807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b35a72c281ef4ead9c6e1ebf89bdffd3" candidate_uuid: "842f35c85ebb41dda94f9909e89de3a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b09f63f9ed1349dfa00cfa58e68377c5"
I20260812 03:57:31.574668 1974 raft_consensus.cc:3037] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:31.574663 1807 raft_consensus.cc:3037] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:31.575490 1974 raft_consensus.cc:2454] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842f35c85ebb41dda94f9909e89de3a4 in term 1.
I20260812 03:57:31.575505 1807 raft_consensus.cc:2454] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842f35c85ebb41dda94f9909e89de3a4 in term 1.
I20260812 03:57:31.575714 1561 leader_election.cc:302] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842f35c85ebb41dda94f9909e89de3a4, a2a3fd59e9a54babb012b2f53e1da667; no voters:
I20260812 03:57:31.575927 2064 raft_consensus.cc:2781] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:31.576117 2064 raft_consensus.cc:686] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 1 LEADER]: Becoming Leader. State: Replica: 842f35c85ebb41dda94f9909e89de3a4, State: Running, Role: LEADER
I20260812 03:57:31.576248 2064 consensus_queue.cc:237] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } }
I20260812 03:57:31.577209 1455 catalog_manager.cc:5495] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193). New cstate: current_term: 1 leader_uuid: "842f35c85ebb41dda94f9909e89de3a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } health_report { overall_health: HEALTHY } } }
Aug 12 03:57:31 dist-test-slave-8tdl krb5kdc[1403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507050, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:31.623592 2077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.615595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37733 (local address 127.0.0.1:33908)
0812 03:57:31.615780 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.615790 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.615861 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.616013 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.616014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.616365 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:31.616418 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.616420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.617082 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.617083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.617625 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.617626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.617968 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.617969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.618017 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.618770 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:31.618775 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:31.620437 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:31.622331 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.622341 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.622344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.622573 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.622799 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:31.622800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:31.622800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:31.622959 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:31.623239 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:31.623248 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:31.623306 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.623379 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.623509 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260812 03:57:31.623673 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.615704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33908 (local address 127.31.43.254:37733)
0812 03:57:31.615811 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.615813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.615854 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:57:31.615922 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.615923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.615934 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.615994 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:31.616495 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.616986 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.617707 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.617896 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.618019 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.620566 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:31.620568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:31.620568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:31.620576 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:31.622239 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.622652 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.622653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.622653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.622732 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:31.623038 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:31.623039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:31.623039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:31.623156 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:31.623216 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.623451 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.623574 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":82}
I20260812 03:57:31.626307 1455 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:33908:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:57:31.626529 1455 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:31.628268 1783 tablet_service.cc:1467] Processing CreateTablet for tablet 8225f044338949558f2abfb0d5332b24 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5f7e2882489344efacf114744283ede2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:57:31.628285 1948 tablet_service.cc:1467] Processing CreateTablet for tablet 8225f044338949558f2abfb0d5332b24 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5f7e2882489344efacf114744283ede2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:57:31.628293 1615 tablet_service.cc:1467] Processing CreateTablet for tablet 8225f044338949558f2abfb0d5332b24 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5f7e2882489344efacf114744283ede2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:57:31.628461 1615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8225f044338949558f2abfb0d5332b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:31.628463 1948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8225f044338949558f2abfb0d5332b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:31.628468 1783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8225f044338949558f2abfb0d5332b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:31.630065 2061 tablet_bootstrap.cc:492] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4: Bootstrap starting.
I20260812 03:57:31.630075 2054 tablet_bootstrap.cc:492] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5: Bootstrap starting.
I20260812 03:57:31.630554 2061 tablet_bootstrap.cc:654] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:31.630784 2054 tablet_bootstrap.cc:654] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:31.631291 2055 tablet_bootstrap.cc:492] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: Bootstrap starting.
I20260812 03:57:31.631561 2054 tablet_bootstrap.cc:492] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5: No bootstrap required, opened a new log
I20260812 03:57:31.631649 2054 ts_tablet_manager.cc:1397] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:57:31.631820 2055 tablet_bootstrap.cc:654] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:31.631816 2054 raft_consensus.cc:348] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.631908 2054 raft_consensus.cc:374] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:31.631925 2054 raft_consensus.cc:729] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b09f63f9ed1349dfa00cfa58e68377c5, State: Initialized, Role: FOLLOWER
I20260812 03:57:31.631983 2054 consensus_queue.cc:260] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.632077 2054 ts_tablet_manager.cc:1428] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:31.632181 2060 tablet_replica.cc:439] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } }
I20260812 03:57:31.632287 2060 tablet_replica.cc:442] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:31.632640 2055 tablet_bootstrap.cc:492] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: No bootstrap required, opened a new log
I20260812 03:57:31.632688 2055 ts_tablet_manager.cc:1397] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:31.632844 2055 raft_consensus.cc:348] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.632937 2055 raft_consensus.cc:374] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:31.632969 2055 raft_consensus.cc:729] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a3fd59e9a54babb012b2f53e1da667, State: Initialized, Role: FOLLOWER
I20260812 03:57:31.633023 2055 consensus_queue.cc:260] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.633143 2055 ts_tablet_manager.cc:1428] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:57:31.633420 2061 tablet_bootstrap.cc:492] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4: No bootstrap required, opened a new log
I20260812 03:57:31.633483 2061 ts_tablet_manager.cc:1397] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 03:57:31.633647 2061 raft_consensus.cc:348] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.633734 2061 raft_consensus.cc:374] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:31.633760 2061 raft_consensus.cc:729] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842f35c85ebb41dda94f9909e89de3a4, State: Initialized, Role: FOLLOWER
I20260812 03:57:31.633836 2061 consensus_queue.cc:260] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.633940 2061 ts_tablet_manager.cc:1428] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:31.634004 2064 tablet_replica.cc:439] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } }
I20260812 03:57:31.634083 2064 tablet_replica.cc:442] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:31.634105 2058 tablet_replica.cc:439] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } }
I20260812 03:57:31.634202 2058 tablet_replica.cc:442] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 03:57:31.649362 2025 tablet.cc:2369] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:57:31.671444 2058 raft_consensus.cc:482] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:31.671566 2058 raft_consensus.cc:504] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.672005 2058 leader_election.cc:288] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193:35569), b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194:35067)
I20260812 03:57:31.675171 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.672286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:35067 (local address 127.31.43.195:45833)
0812 03:57:31.672361 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.672376 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.672468 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.672648 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.672649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.672651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:31.672736 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.672738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.673476 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.673477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.674268 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.674270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.674849 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.674850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.674960 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.674963 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.675033 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.675080 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 03:57:31.675159 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.672448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:45833 (local address 127.31.43.194:35067)
0812 03:57:31.672513 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.672516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.672524 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:57:31.672555 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.672556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.672565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.672644 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:31.672817 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.673360 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.674380 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.674764 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.674850 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.674861 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.675042 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.675081 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:57:31.675450 1807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8225f044338949558f2abfb0d5332b24" candidate_uuid: "a2a3fd59e9a54babb012b2f53e1da667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" is_pre_election: true
I20260812 03:57:31.675554 1807 raft_consensus.cc:2454] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2a3fd59e9a54babb012b2f53e1da667 in term 0.
I20260812 03:57:31.675558 2082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.672286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:35569 (local address 127.31.43.195:59215)
0812 03:57:31.672555 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.672564 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.672647 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.672804 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.672805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.672806 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:31.672974 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.672976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.673775 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.673776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.674525 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.674527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.675199 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.675200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.675292 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.675297 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.675402 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.675446 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":60,"thread_start_us":145,"threads_started":1}
I20260812 03:57:31.675536 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.672414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:59215 (local address 127.31.43.193:35569)
0812 03:57:31.672466 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.672468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.672638 (+ 170us) server_negotiation.cc:407] Connection header received
0812 03:57:31.672707 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.672708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.672717 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.672789 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:31.673058 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.673687 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.674620 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.675098 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.675194 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.675203 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.675419 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.675464 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:57:31.675742 1893 leader_election.cc:302] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2a3fd59e9a54babb012b2f53e1da667, b09f63f9ed1349dfa00cfa58e68377c5; no voters:
I20260812 03:57:31.675953 2058 raft_consensus.cc:2781] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:31.675938 1638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8225f044338949558f2abfb0d5332b24" candidate_uuid: "a2a3fd59e9a54babb012b2f53e1da667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842f35c85ebb41dda94f9909e89de3a4" is_pre_election: true
I20260812 03:57:31.676028 2058 raft_consensus.cc:482] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:31.676059 2058 raft_consensus.cc:3037] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:31.676090 1638 raft_consensus.cc:2454] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2a3fd59e9a54babb012b2f53e1da667 in term 0.
I20260812 03:57:31.676812 2058 raft_consensus.cc:504] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.677043 2058 leader_election.cc:288] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [CANDIDATE]: Term 1 election: Requested vote from peers 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193:35569), b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194:35067)
I20260812 03:57:31.677261 1807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8225f044338949558f2abfb0d5332b24" candidate_uuid: "a2a3fd59e9a54babb012b2f53e1da667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b09f63f9ed1349dfa00cfa58e68377c5"
I20260812 03:57:31.677276 1638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8225f044338949558f2abfb0d5332b24" candidate_uuid: "a2a3fd59e9a54babb012b2f53e1da667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842f35c85ebb41dda94f9909e89de3a4"
I20260812 03:57:31.677366 1807 raft_consensus.cc:3037] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:31.677386 1638 raft_consensus.cc:3037] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:31.678092 1807 raft_consensus.cc:2454] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2a3fd59e9a54babb012b2f53e1da667 in term 1.
I20260812 03:57:31.678160 1638 raft_consensus.cc:2454] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2a3fd59e9a54babb012b2f53e1da667 in term 1.
I20260812 03:57:31.678309 1893 leader_election.cc:302] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2a3fd59e9a54babb012b2f53e1da667, b09f63f9ed1349dfa00cfa58e68377c5; no voters:
I20260812 03:57:31.678418 2058 raft_consensus.cc:2781] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:31.678658 2058 raft_consensus.cc:686] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [term 1 LEADER]: Becoming Leader. State: Replica: a2a3fd59e9a54babb012b2f53e1da667, State: Running, Role: LEADER
I20260812 03:57:31.678819 2058 consensus_queue.cc:237] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.679433 2083 tablet_replica.cc:439] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: TxnStatusTablet state changed. Reason: New leader a2a3fd59e9a54babb012b2f53e1da667. Latest consensus state: current_term: 1 leader_uuid: "a2a3fd59e9a54babb012b2f53e1da667" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } }
I20260812 03:57:31.679538 2083 tablet_replica.cc:442] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:57:31.679754 2087 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:57:31.679875 2087 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:57:31.679737 1454 catalog_manager.cc:5495] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195). New cstate: current_term: 1 leader_uuid: "a2a3fd59e9a54babb012b2f53e1da667" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } health_report { overall_health: HEALTHY } } }
I20260812 03:57:31.698110 2077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.694971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:42951 (local address 127.0.0.1:53472)
0812 03:57:31.695040 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.695054 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.695124 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.695309 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.695309 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.695311 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:31.695375 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.695377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.696016 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.696017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.696999 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.697003 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.697403 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.697404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.697486 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.697514 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:31.697867 (+ 353us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:31.697878 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.697955 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.698016 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 03:57:31.698117 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.694995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53472 (local address 127.31.43.195:42951)
0812 03:57:31.695048 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.695052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.695113 (+ 61us) server_negotiation.cc:407] Connection header received
0812 03:57:31.695212 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.695213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.695231 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.695292 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:31.695442 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.695932 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.697113 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.697312 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.697402 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.697665 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:31.697783 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:31.697862 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.697991 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.698019 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:57:31.700875 1638 raft_consensus.cc:1261] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Refusing update from remote peer a2a3fd59e9a54babb012b2f53e1da667: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:31.700878 1807 raft_consensus.cc:1261] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Refusing update from remote peer a2a3fd59e9a54babb012b2f53e1da667: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:31.701283 2058 consensus_queue.cc:1035] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:31.701472 2083 consensus_queue.cc:1035] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:31.702809 2064 tablet_replica.cc:439] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4: TxnStatusTablet state changed. Reason: New leader a2a3fd59e9a54babb012b2f53e1da667. Latest consensus state: current_term: 1 leader_uuid: "a2a3fd59e9a54babb012b2f53e1da667" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } }
I20260812 03:57:31.702911 2064 tablet_replica.cc:442] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:31.703097 2060 tablet_replica.cc:439] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5: TxnStatusTablet state changed. Reason: New leader a2a3fd59e9a54babb012b2f53e1da667. Latest consensus state: current_term: 1 leader_uuid: "a2a3fd59e9a54babb012b2f53e1da667" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } }
I20260812 03:57:31.703208 2060 tablet_replica.cc:442] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:31.704020 2083 tablet_replica.cc:439] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2a3fd59e9a54babb012b2f53e1da667" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } }
I20260812 03:57:31.704121 2083 tablet_replica.cc:442] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:57:31.704257 2058 tablet_replica.cc:439] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2a3fd59e9a54babb012b2f53e1da667" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } }
I20260812 03:57:31.704317 2058 tablet_replica.cc:442] T 8225f044338949558f2abfb0d5332b24 P a2a3fd59e9a54babb012b2f53e1da667: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:57:31.704566 2090 mvcc.cc:205] Tried to move back new op lower bound from 7317532883763826688 to 7317532883677253632. Current Snapshot: MvccSnapshot[applied={T|T < 7317532883763826688}]
I20260812 03:57:31.704913 2064 tablet_replica.cc:439] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2a3fd59e9a54babb012b2f53e1da667" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } }
I20260812 03:57:31.705019 2064 tablet_replica.cc:442] T 8225f044338949558f2abfb0d5332b24 P 842f35c85ebb41dda94f9909e89de3a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:31.706046 2060 tablet_replica.cc:439] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2a3fd59e9a54babb012b2f53e1da667" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } } }
I20260812 03:57:31.706159 2060 tablet_replica.cc:442] T 8225f044338949558f2abfb0d5332b24 P b09f63f9ed1349dfa00cfa58e68377c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:31.710948 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.708178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48160 (local address 127.31.43.193:35569)
0812 03:57:31.708222 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.708225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.708234 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:31.708312 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.708313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.708334 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.708430 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:31.708611 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.709142 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.709999 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.710237 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.710331 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.710525 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:31.710659 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:31.710805 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.710825 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.710854 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:57:31.717201 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.714168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:33297 (local address 127.31.43.195:42951)
0812 03:57:31.714238 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.714241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.714416 (+ 175us) server_negotiation.cc:407] Connection header received
0812 03:57:31.714491 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.714492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.714502 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.714548 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:31.714740 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.715250 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.716441 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.716648 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.716710 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.716956 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:31.716991 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:31.717030 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.717114 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.717132 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:57:31.717278 2104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.714069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:42951 (local address 127.31.43.193:33297)
0812 03:57:31.714331 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.714343 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.714419 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.714561 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.714562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.714563 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:31.714643 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.714644 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.715344 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.715345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.716326 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.716330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.716732 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.716734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.716809 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.716834 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:31.717054 (+ 220us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:31.717057 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.717113 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.717164 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":115,"threads_started":1}
I20260812 03:57:31.720165 1807 raft_consensus.cc:1261] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Refusing update from remote peer 842f35c85ebb41dda94f9909e89de3a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:31.720525 2067 consensus_queue.cc:1035] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:31.726150 1974 raft_consensus.cc:1261] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Refusing update from remote peer 842f35c85ebb41dda94f9909e89de3a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:31.727880 2067 consensus_queue.cc:1035] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:31.729807 2093 mvcc.cc:205] Tried to move back new op lower bound from 7317532883843522560 to 7317532883257016320. Current Snapshot: MvccSnapshot[applied={T|T < 7317532883843522560}]
I20260812 03:57:31.730785 2092 mvcc.cc:205] Tried to move back new op lower bound from 7317532883843522560 to 7317532883257016320. Current Snapshot: MvccSnapshot[applied={T|T < 7317532883843522560}]
I20260812 03:57:31.739311 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.736227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:35569 (local address 127.31.43.195:55705)
0812 03:57:31.736588 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.736602 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.736659 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.736819 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.736820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.736821 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:31.736900 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.736902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.737729 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.737730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.738328 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.738329 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.738779 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.738780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.738853 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.738881 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:31.739139 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:31.739141 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.739184 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.739234 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298}
I20260812 03:57:31.739602 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.736427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:55705 (local address 127.31.43.193:35569)
0812 03:57:31.736504 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.736506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.736653 (+ 147us) server_negotiation.cc:407] Connection header received
0812 03:57:31.736724 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.736724 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.736734 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.736812 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:31.736988 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.737651 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.738425 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.738667 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.738780 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.739000 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:31.739045 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:31.739109 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.739504 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.739532 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:57:31.750916 1454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:31.751083 1454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:31.752184 1454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97: Sending DeleteTablet for 3 replicas of tablet b35a72c281ef4ead9c6e1ebf89bdffd3
I20260812 03:57:31.752687 1948 tablet_service.cc:1514] Processing DeleteTablet for tablet b35a72c281ef4ead9c6e1ebf89bdffd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:31 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:53466
I20260812 03:57:31.752764 1783 tablet_service.cc:1514] Processing DeleteTablet for tablet b35a72c281ef4ead9c6e1ebf89bdffd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:31 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:38492
I20260812 03:57:31.753203 1615 tablet_service.cc:1514] Processing DeleteTablet for tablet b35a72c281ef4ead9c6e1ebf89bdffd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:31 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:48144
I20260812 03:57:31.753521 2117 tablet_replica.cc:330] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4: stopping tablet replica
I20260812 03:57:31.753717 2117 raft_consensus.cc:2229] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:57:31.753844 1454 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:31.754033 1454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:31.753943 2117 raft_consensus.cc:2258] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:31.754555 2115 tablet_replica.cc:330] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5: stopping tablet replica
I20260812 03:57:31.755501 2117 ts_tablet_manager.cc:1905] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:31.756129 1783 tablet_service.cc:1467] Processing CreateTablet for tablet 714a921b1e144e19954f00db033b9d3c (DEFAULT_TABLE table=test-table [id=2d9f625185854c81961a66560dac5c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:31.756290 1783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714a921b1e144e19954f00db033b9d3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:31.757973 2054 tablet_bootstrap.cc:492] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5: Bootstrap starting.
I20260812 03:57:31.758054 1948 tablet_service.cc:1467] Processing CreateTablet for tablet 714a921b1e144e19954f00db033b9d3c (DEFAULT_TABLE table=test-table [id=2d9f625185854c81961a66560dac5c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:31.758203 1948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714a921b1e144e19954f00db033b9d3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:31.758524 2054 tablet_bootstrap.cc:654] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:31.759166 2117 ts_tablet_manager.cc:1918] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:31.759371 2117 log.cc:1196] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/wal/wals/b35a72c281ef4ead9c6e1ebf89bdffd3
I20260812 03:57:31.759850 2117 ts_tablet_manager.cc:1939] T b35a72c281ef4ead9c6e1ebf89bdffd3 P 842f35c85ebb41dda94f9909e89de3a4: Deleting consensus metadata
I20260812 03:57:31.760063 2055 tablet_bootstrap.cc:492] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667: Bootstrap starting.
I20260812 03:57:31.760453 2055 tablet_bootstrap.cc:654] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:31.760639 1615 tablet_service.cc:1467] Processing CreateTablet for tablet 714a921b1e144e19954f00db033b9d3c (DEFAULT_TABLE table=test-table [id=2d9f625185854c81961a66560dac5c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:31.760696 1434 catalog_manager.cc:4841] TS 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193:35569): tablet b35a72c281ef4ead9c6e1ebf89bdffd3 (table test-table [id=74aad9e1758b4abda99fcbc6f0e42649]) successfully deleted
I20260812 03:57:31.760776 1615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714a921b1e144e19954f00db033b9d3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:31.760833 2116 tablet_replica.cc:330] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667: stopping tablet replica
I20260812 03:57:31.760959 2116 raft_consensus.cc:2229] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:31.761021 2116 raft_consensus.cc:2258] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:31.761185 2115 raft_consensus.cc:2229] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:31.761262 2115 raft_consensus.cc:2258] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:31.761776 2115 ts_tablet_manager.cc:1905] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:31.761807 2116 ts_tablet_manager.cc:1905] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:31.762338 2061 tablet_bootstrap.cc:492] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4: Bootstrap starting.
I20260812 03:57:31.762894 2061 tablet_bootstrap.cc:654] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:31.764082 2061 tablet_bootstrap.cc:492] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4: No bootstrap required, opened a new log
I20260812 03:57:31.764145 2061 ts_tablet_manager.cc:1397] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:31.764324 2061 raft_consensus.cc:348] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.764415 2061 raft_consensus.cc:374] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:31.764438 2061 raft_consensus.cc:729] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842f35c85ebb41dda94f9909e89de3a4, State: Initialized, Role: FOLLOWER
I20260812 03:57:31.764485 2061 consensus_queue.cc:260] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.764593 2061 ts_tablet_manager.cc:1428] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:31.765355 2054 tablet_bootstrap.cc:492] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5: No bootstrap required, opened a new log
I20260812 03:57:31.765421 2054 ts_tablet_manager.cc:1397] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260812 03:57:31.765599 2054 raft_consensus.cc:348] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.765692 2054 raft_consensus.cc:374] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:31.765713 2054 raft_consensus.cc:729] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b09f63f9ed1349dfa00cfa58e68377c5, State: Initialized, Role: FOLLOWER
I20260812 03:57:31.765774 2054 consensus_queue.cc:260] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.765894 2054 ts_tablet_manager.cc:1428] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:31.765942 2115 ts_tablet_manager.cc:1918] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:31.766023 2115 log.cc:1196] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/b35a72c281ef4ead9c6e1ebf89bdffd3
I20260812 03:57:31.766366 2115 ts_tablet_manager.cc:1939] T b35a72c281ef4ead9c6e1ebf89bdffd3 P b09f63f9ed1349dfa00cfa58e68377c5: Deleting consensus metadata
I20260812 03:57:31.767386 1438 catalog_manager.cc:4841] TS b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194:35067): tablet b35a72c281ef4ead9c6e1ebf89bdffd3 (table test-table [id=74aad9e1758b4abda99fcbc6f0e42649]) successfully deleted
I20260812 03:57:31.770888 2116 ts_tablet_manager.cc:1918] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:31.771013 2116 log.cc:1196] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/b35a72c281ef4ead9c6e1ebf89bdffd3
I20260812 03:57:31.771291 2055 tablet_bootstrap.cc:492] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667: No bootstrap required, opened a new log
I20260812 03:57:31.771343 2116 ts_tablet_manager.cc:1939] T b35a72c281ef4ead9c6e1ebf89bdffd3 P a2a3fd59e9a54babb012b2f53e1da667: Deleting consensus metadata
I20260812 03:57:31.771384 2055 ts_tablet_manager.cc:1397] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260812 03:57:31.771581 2055 raft_consensus.cc:348] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.771692 2055 raft_consensus.cc:374] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:31.771732 2055 raft_consensus.cc:729] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a3fd59e9a54babb012b2f53e1da667, State: Initialized, Role: FOLLOWER
I20260812 03:57:31.771786 2055 consensus_queue.cc:260] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.771903 2055 ts_tablet_manager.cc:1428] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:57:31.771899 1436 catalog_manager.cc:4841] TS a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951): tablet b35a72c281ef4ead9c6e1ebf89bdffd3 (table test-table [id=74aad9e1758b4abda99fcbc6f0e42649]) successfully deleted
I20260812 03:57:31.840687 2060 raft_consensus.cc:482] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:31.840837 2060 raft_consensus.cc:504] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.841212 2060 leader_election.cc:288] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193:35569), a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951)
I20260812 03:57:31.845558 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.841692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:35693 (local address 127.31.43.195:42951)
0812 03:57:31.841762 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.841765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.841860 (+ 95us) server_negotiation.cc:407] Connection header received
0812 03:57:31.841952 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.841953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.841965 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.842031 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:31.842244 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.843003 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.844563 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.845116 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.845172 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.845182 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.845431 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.845474 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:57:31.845706 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.841693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:46807 (local address 127.31.43.193:35569)
0812 03:57:31.841803 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.841805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.842002 (+ 197us) server_negotiation.cc:407] Connection header received
0812 03:57:31.842111 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.842112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.842126 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.842169 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:31.842451 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.843193 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.844682 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.845257 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.845309 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.845317 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.845591 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.845631 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 03:57:31.845983 2122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.841480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:35569 (local address 127.31.43.194:46807)
0812 03:57:31.841945 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.841954 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.842029 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.842249 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.842250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.842252 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:31.842354 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.842355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.843304 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.843304 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.844555 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.844558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.845408 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.845409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.845493 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.845497 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.845845 (+ 348us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.845897 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":96,"threads_started":1}
I20260812 03:57:31.846290 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.841475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:42951 (local address 127.31.43.194:35693)
0812 03:57:31.841757 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.841770 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.841857 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.842089 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.842090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.842093 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:31.842152 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.842154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.843118 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.843119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.844431 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.844436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.845256 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.845256 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.845326 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.845330 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.846027 (+ 697us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.846201 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":100,"threads_started":1}
I20260812 03:57:31.846242 1638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714a921b1e144e19954f00db033b9d3c" candidate_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842f35c85ebb41dda94f9909e89de3a4" is_pre_election: true
I20260812 03:57:31.846354 1638 raft_consensus.cc:2454] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b09f63f9ed1349dfa00cfa58e68377c5 in term 0.
I20260812 03:57:31.846560 1727 leader_election.cc:302] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842f35c85ebb41dda94f9909e89de3a4, b09f63f9ed1349dfa00cfa58e68377c5; no voters:
I20260812 03:57:31.846740 2060 raft_consensus.cc:2781] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:31.846776 2060 raft_consensus.cc:482] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:31.846798 2060 raft_consensus.cc:3037] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:31.846778 1974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714a921b1e144e19954f00db033b9d3c" candidate_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a3fd59e9a54babb012b2f53e1da667" is_pre_election: true
I20260812 03:57:31.846865 1974 raft_consensus.cc:2454] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b09f63f9ed1349dfa00cfa58e68377c5 in term 0.
I20260812 03:57:31.847764 2060 raft_consensus.cc:504] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.847927 2060 leader_election.cc:288] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [CANDIDATE]: Term 1 election: Requested vote from peers 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193:35569), a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951)
I20260812 03:57:31.848124 1974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714a921b1e144e19954f00db033b9d3c" candidate_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a3fd59e9a54babb012b2f53e1da667"
I20260812 03:57:31.848217 1974 raft_consensus.cc:3037] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:31.848186 1638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714a921b1e144e19954f00db033b9d3c" candidate_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842f35c85ebb41dda94f9909e89de3a4"
I20260812 03:57:31.848269 1638 raft_consensus.cc:3037] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:31.849129 1974 raft_consensus.cc:2454] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b09f63f9ed1349dfa00cfa58e68377c5 in term 1.
I20260812 03:57:31.849136 1638 raft_consensus.cc:2454] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b09f63f9ed1349dfa00cfa58e68377c5 in term 1.
I20260812 03:57:31.849318 1729 leader_election.cc:302] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2a3fd59e9a54babb012b2f53e1da667, b09f63f9ed1349dfa00cfa58e68377c5; no voters:
I20260812 03:57:31.849403 2060 raft_consensus.cc:2781] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:31.849566 2060 raft_consensus.cc:686] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 LEADER]: Becoming Leader. State: Replica: b09f63f9ed1349dfa00cfa58e68377c5, State: Running, Role: LEADER
I20260812 03:57:31.849651 2060 consensus_queue.cc:237] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:31.850467 1454 catalog_manager.cc:5495] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194). New cstate: current_term: 1 leader_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } health_report { overall_health: UNKNOWN } } }
I20260812 03:57:31.862476 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.859076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38504 (local address 127.31.43.194:35067)
0812 03:57:31.859124 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.859126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.859137 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:57:31.859204 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.859205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.859233 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.859316 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:31.859498 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.860330 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.861478 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.861774 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.861856 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.862051 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:31.862189 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:31.862262 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.862333 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.862363 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:57:31.869135 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.865837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:48307 (local address 127.31.43.195:42951)
0812 03:57:31.865888 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.865890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.866056 (+ 166us) server_negotiation.cc:407] Connection header received
0812 03:57:31.866098 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.866100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.866113 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.866180 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:31.866351 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.867135 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.868201 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.868516 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.868601 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.868831 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:31.868871 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:31.868936 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.869007 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.869046 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:57:31.869175 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.865780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:42951 (local address 127.31.43.194:48307)
0812 03:57:31.865966 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.865977 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.866045 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.866189 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.866190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.866192 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:31.866277 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.866279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.867240 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.867242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.868090 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.868094 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.868618 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.868620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.868695 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.868724 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:31.868948 (+ 224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:31.868951 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.868993 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.869048 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260812 03:57:31.872265 1638 raft_consensus.cc:1261] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Refusing update from remote peer b09f63f9ed1349dfa00cfa58e68377c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:31.872249 1974 raft_consensus.cc:1261] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Refusing update from remote peer b09f63f9ed1349dfa00cfa58e68377c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:31.872603 2124 consensus_queue.cc:1035] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:31.872754 2060 consensus_queue.cc:1035] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:31.881454 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.878619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:35067 (local address 127.31.43.195:54985)
0812 03:57:31.878671 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:31.878710 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:31.878752 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:31.878902 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:31.878903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:31.878904 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:31.878957 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.878959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.879866 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.879866 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.880405 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:31.880407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.880886 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.880887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:31.880949 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.880955 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:31.881264 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:31.881266 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:57:31.881310 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 03:57:31.881366 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 03:57:31.881726 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:31.878665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:54985 (local address 127.31.43.194:35067)
0812 03:57:31.878738 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:31.878740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:31.878749 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:31.878819 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:31.878820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:31.878830 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:31.878892 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:31.879026 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.879771 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.880503 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:31.880771 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:31.880847 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:31.881102 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:31.881143 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:31.881197 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:31.881625 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 03:57:31.881652 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 03:57:31.882081 2092 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 03:57:33.415539 1455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:33.415683 1455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:33.416728 1455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97: Sending DeleteTablet for 3 replicas of tablet 714a921b1e144e19954f00db033b9d3c
I20260812 03:57:33.417083 1948 tablet_service.cc:1514] Processing DeleteTablet for tablet 714a921b1e144e19954f00db033b9d3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:33 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:53466
I20260812 03:57:33.417100 1783 tablet_service.cc:1514] Processing DeleteTablet for tablet 714a921b1e144e19954f00db033b9d3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:33 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:38492
I20260812 03:57:33.417145 1615 tablet_service.cc:1514] Processing DeleteTablet for tablet 714a921b1e144e19954f00db033b9d3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:33 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:48144
I20260812 03:57:33.420711 2162 tablet_replica.cc:330] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4: stopping tablet replica
I20260812 03:57:33.420867 2162 raft_consensus.cc:2229] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:33.420953 2162 raft_consensus.cc:2258] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:33.421327 2162 ts_tablet_manager.cc:1905] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:33.421919 2160 tablet_replica.cc:330] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667: stopping tablet replica
I20260812 03:57:33.421998 2160 raft_consensus.cc:2229] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:33.422066 2160 raft_consensus.cc:2258] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:33.422202 2161 tablet_replica.cc:330] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5: stopping tablet replica
I20260812 03:57:33.422335 2161 raft_consensus.cc:2229] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:57:33.422356 2160 ts_tablet_manager.cc:1905] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:33.422479 2161 raft_consensus.cc:2258] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:33.422773 2161 ts_tablet_manager.cc:1905] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:33.423352 2162 ts_tablet_manager.cc:1918] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:57:33.423473 2162 log.cc:1196] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/wal/wals/714a921b1e144e19954f00db033b9d3c
I20260812 03:57:33.423830 2162 ts_tablet_manager.cc:1939] T 714a921b1e144e19954f00db033b9d3c P 842f35c85ebb41dda94f9909e89de3a4: Deleting consensus metadata
I20260812 03:57:33.424338 1434 catalog_manager.cc:4841] TS 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193:35569): tablet 714a921b1e144e19954f00db033b9d3c (table test-table [id=2d9f625185854c81961a66560dac5c57]) successfully deleted
I20260812 03:57:33.424628 2160 ts_tablet_manager.cc:1918] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:57:33.424711 2160 log.cc:1196] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/714a921b1e144e19954f00db033b9d3c
I20260812 03:57:33.424748 2161 ts_tablet_manager.cc:1918] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:57:33.424844 2161 log.cc:1196] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/714a921b1e144e19954f00db033b9d3c
I20260812 03:57:33.424964 2160 ts_tablet_manager.cc:1939] T 714a921b1e144e19954f00db033b9d3c P a2a3fd59e9a54babb012b2f53e1da667: Deleting consensus metadata
I20260812 03:57:33.425125 2161 ts_tablet_manager.cc:1939] T 714a921b1e144e19954f00db033b9d3c P b09f63f9ed1349dfa00cfa58e68377c5: Deleting consensus metadata
I20260812 03:57:33.425550 1436 catalog_manager.cc:4841] TS a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951): tablet 714a921b1e144e19954f00db033b9d3c (table test-table [id=2d9f625185854c81961a66560dac5c57]) successfully deleted
I20260812 03:57:33.425905 1438 catalog_manager.cc:4841] TS b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194:35067): tablet 714a921b1e144e19954f00db033b9d3c (table test-table [id=2d9f625185854c81961a66560dac5c57]) successfully deleted
Aug 12 03:57:33 dist-test-slave-8tdl krb5kdc[1403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507053, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:57:33 dist-test-slave-8tdl krb5kdc[1403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507053, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:33.452872 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:33.439664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33910 (local address 127.31.43.254:37733)
0812 03:57:33.439847 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:33.439849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:33.439858 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:33.439886 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:33.439887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:33.439898 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:33.439971 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:33.440600 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:33.441166 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:33.441919 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:33.442151 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:33.442210 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:33.449907 (+ 7697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:33.449910 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:33.449910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:33.449919 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:33.451531 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:33.451935 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:33.451937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:33.451937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:33.452001 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:33.452238 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:33.452239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:33.452239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:33.452366 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:33.452474 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:33.452626 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:57:33.452733 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":103,"threads_started":1}
I20260812 03:57:33.457785 1455 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:33.457971 1455 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:33.459700 1615 tablet_service.cc:1467] Processing CreateTablet for tablet c3934fb7b527497393e3cc4be252d3ca (DEFAULT_TABLE table=test-table [id=f11521a2fc2f4cc4b1a6f7cfea8f28cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:33.459702 1783 tablet_service.cc:1467] Processing CreateTablet for tablet c3934fb7b527497393e3cc4be252d3ca (DEFAULT_TABLE table=test-table [id=f11521a2fc2f4cc4b1a6f7cfea8f28cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:33.459748 1948 tablet_service.cc:1467] Processing CreateTablet for tablet c3934fb7b527497393e3cc4be252d3ca (DEFAULT_TABLE table=test-table [id=f11521a2fc2f4cc4b1a6f7cfea8f28cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:33.459872 1783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3934fb7b527497393e3cc4be252d3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:33.459872 1948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3934fb7b527497393e3cc4be252d3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:33.459885 1615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3934fb7b527497393e3cc4be252d3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:33.461012 2174 tablet_bootstrap.cc:492] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4: Bootstrap starting.
I20260812 03:57:33.461503 2177 tablet_bootstrap.cc:492] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667: Bootstrap starting.
I20260812 03:57:33.461531 2174 tablet_bootstrap.cc:654] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:33.461580 2176 tablet_bootstrap.cc:492] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5: Bootstrap starting.
I20260812 03:57:33.461971 2177 tablet_bootstrap.cc:654] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:33.461972 2176 tablet_bootstrap.cc:654] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:33.462288 2174 tablet_bootstrap.cc:492] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4: No bootstrap required, opened a new log
I20260812 03:57:33.462352 2174 ts_tablet_manager.cc:1397] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:33.462503 2174 raft_consensus.cc:348] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.462594 2174 raft_consensus.cc:374] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:33.462610 2174 raft_consensus.cc:729] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842f35c85ebb41dda94f9909e89de3a4, State: Initialized, Role: FOLLOWER
I20260812 03:57:33.462666 2174 consensus_queue.cc:260] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.462716 2176 tablet_bootstrap.cc:492] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5: No bootstrap required, opened a new log
I20260812 03:57:33.462769 2176 ts_tablet_manager.cc:1397] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:33.462862 2174 ts_tablet_manager.cc:1428] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:57:33.462900 2176 raft_consensus.cc:348] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.463002 2176 raft_consensus.cc:374] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:33.463032 2176 raft_consensus.cc:729] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b09f63f9ed1349dfa00cfa58e68377c5, State: Initialized, Role: FOLLOWER
I20260812 03:57:33.463078 2176 consensus_queue.cc:260] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.463195 2176 ts_tablet_manager.cc:1428] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:33.463323 2177 tablet_bootstrap.cc:492] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667: No bootstrap required, opened a new log
I20260812 03:57:33.463372 2177 ts_tablet_manager.cc:1397] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:33.463496 2177 raft_consensus.cc:348] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.463580 2177 raft_consensus.cc:374] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:33.463632 2177 raft_consensus.cc:729] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a3fd59e9a54babb012b2f53e1da667, State: Initialized, Role: FOLLOWER
I20260812 03:57:33.463719 2177 consensus_queue.cc:260] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.463898 2177 ts_tablet_manager.cc:1428] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:57:33.487275 2060 raft_consensus.cc:482] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:33.487402 2060 raft_consensus.cc:504] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.487619 2060 leader_election.cc:288] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193:35569), a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951)
I20260812 03:57:33.487905 1638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3934fb7b527497393e3cc4be252d3ca" candidate_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842f35c85ebb41dda94f9909e89de3a4" is_pre_election: true
I20260812 03:57:33.487896 1974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3934fb7b527497393e3cc4be252d3ca" candidate_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a3fd59e9a54babb012b2f53e1da667" is_pre_election: true
I20260812 03:57:33.488009 1974 raft_consensus.cc:2454] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b09f63f9ed1349dfa00cfa58e68377c5 in term 0.
I20260812 03:57:33.488011 1638 raft_consensus.cc:2454] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b09f63f9ed1349dfa00cfa58e68377c5 in term 0.
I20260812 03:57:33.488205 1729 leader_election.cc:302] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2a3fd59e9a54babb012b2f53e1da667, b09f63f9ed1349dfa00cfa58e68377c5; no voters:
I20260812 03:57:33.488315 2060 raft_consensus.cc:2781] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:33.488365 2060 raft_consensus.cc:482] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:33.488386 2060 raft_consensus.cc:3037] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:33.489264 2060 raft_consensus.cc:504] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.489427 2060 leader_election.cc:288] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [CANDIDATE]: Term 1 election: Requested vote from peers 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193:35569), a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951)
I20260812 03:57:33.489583 1638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3934fb7b527497393e3cc4be252d3ca" candidate_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842f35c85ebb41dda94f9909e89de3a4"
I20260812 03:57:33.489670 1638 raft_consensus.cc:3037] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:33.489647 1974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3934fb7b527497393e3cc4be252d3ca" candidate_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a3fd59e9a54babb012b2f53e1da667"
I20260812 03:57:33.489743 1974 raft_consensus.cc:3037] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:33.490419 1638 raft_consensus.cc:2454] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b09f63f9ed1349dfa00cfa58e68377c5 in term 1.
I20260812 03:57:33.490439 1974 raft_consensus.cc:2454] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b09f63f9ed1349dfa00cfa58e68377c5 in term 1.
I20260812 03:57:33.490622 1727 leader_election.cc:302] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842f35c85ebb41dda94f9909e89de3a4, b09f63f9ed1349dfa00cfa58e68377c5; no voters:
I20260812 03:57:33.490814 2060 raft_consensus.cc:2781] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:33.490916 2060 raft_consensus.cc:686] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 LEADER]: Becoming Leader. State: Replica: b09f63f9ed1349dfa00cfa58e68377c5, State: Running, Role: LEADER
I20260812 03:57:33.490989 2060 consensus_queue.cc:237] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.491766 1455 catalog_manager.cc:5495] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194). New cstate: current_term: 1 leader_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } health_report { overall_health: UNKNOWN } } }
I20260812 03:57:33.504721 2186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:33.502085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38516 (local address 127.31.43.194:35067)
0812 03:57:33.502282 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:33.502284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:33.502292 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:57:33.502317 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:33.502318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:33.502331 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:33.502396 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:33.502564 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:33.503156 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:33.503947 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:33.504168 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:33.504246 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:33.504413 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:33.504449 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:33.504525 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:33.504587 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 03:57:33.504610 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":134,"threads_started":1}
I20260812 03:57:33.507491 1974 raft_consensus.cc:1261] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Refusing update from remote peer b09f63f9ed1349dfa00cfa58e68377c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:33.507504 1638 raft_consensus.cc:1261] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Refusing update from remote peer b09f63f9ed1349dfa00cfa58e68377c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:33.507740 2060 consensus_queue.cc:1035] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:33.507853 2154 consensus_queue.cc:1035] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:33.518137 1455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:33.518257 1455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:33.518999 1455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97: Sending DeleteTablet for 3 replicas of tablet c3934fb7b527497393e3cc4be252d3ca
I20260812 03:57:33.519330 1783 tablet_service.cc:1514] Processing DeleteTablet for tablet c3934fb7b527497393e3cc4be252d3ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:33 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:38492
I20260812 03:57:33.519490 1615 tablet_service.cc:1514] Processing DeleteTablet for tablet c3934fb7b527497393e3cc4be252d3ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:33 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:48144
I20260812 03:57:33.519507 2161 tablet_replica.cc:330] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5: stopping tablet replica
I20260812 03:57:33.519608 2161 raft_consensus.cc:2229] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:57:33.519605 1948 tablet_service.cc:1514] Processing DeleteTablet for tablet c3934fb7b527497393e3cc4be252d3ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:33 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:53466
I20260812 03:57:33.519620 2162 tablet_replica.cc:330] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4: stopping tablet replica
I20260812 03:57:33.519681 2161 raft_consensus.cc:2258] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:33.519688 2162 raft_consensus.cc:2229] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:33.519693 2160 tablet_replica.cc:330] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667: stopping tablet replica
I20260812 03:57:33.519735 2162 raft_consensus.cc:2258] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:33.519749 2160 raft_consensus.cc:2229] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:33.519790 2160 raft_consensus.cc:2258] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:33.520010 1455 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:57:33.520133 2161 ts_tablet_manager.cc:1905] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:33.520200 2162 ts_tablet_manager.cc:1905] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:57:33.520221 1455 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:33.521528 1783 tablet_service.cc:1467] Processing CreateTablet for tablet ea2b768c98164c7a851aafb199d63f8e (DEFAULT_TABLE table=test-table [id=b693ce883e6a49f4b08004e6fa380ee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:33.521682 1783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea2b768c98164c7a851aafb199d63f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:33.521821 1615 tablet_service.cc:1467] Processing CreateTablet for tablet ea2b768c98164c7a851aafb199d63f8e (DEFAULT_TABLE table=test-table [id=b693ce883e6a49f4b08004e6fa380ee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:33.522056 2161 ts_tablet_manager.cc:1918] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:33.522081 1948 tablet_service.cc:1467] Processing CreateTablet for tablet ea2b768c98164c7a851aafb199d63f8e (DEFAULT_TABLE table=test-table [id=b693ce883e6a49f4b08004e6fa380ee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:33.522089 1615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea2b768c98164c7a851aafb199d63f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:33.522114 2161 log.cc:1196] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/c3934fb7b527497393e3cc4be252d3ca
I20260812 03:57:33.522286 1948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea2b768c98164c7a851aafb199d63f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:33.522393 2161 ts_tablet_manager.cc:1939] T c3934fb7b527497393e3cc4be252d3ca P b09f63f9ed1349dfa00cfa58e68377c5: Deleting consensus metadata
I20260812 03:57:33.522740 1438 catalog_manager.cc:4841] TS b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194:35067): tablet c3934fb7b527497393e3cc4be252d3ca (table test-table [id=f11521a2fc2f4cc4b1a6f7cfea8f28cd]) successfully deleted
I20260812 03:57:33.523373 2176 tablet_bootstrap.cc:492] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5: Bootstrap starting.
I20260812 03:57:33.523388 2174 tablet_bootstrap.cc:492] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4: Bootstrap starting.
I20260812 03:57:33.523684 2177 tablet_bootstrap.cc:492] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667: Bootstrap starting.
I20260812 03:57:33.523758 2176 tablet_bootstrap.cc:654] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:33.523828 2174 tablet_bootstrap.cc:654] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:33.523880 2160 ts_tablet_manager.cc:1905] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:33.524169 2177 tablet_bootstrap.cc:654] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:33.522089 2162 ts_tablet_manager.cc:1918] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:33.524531 2162 log.cc:1196] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-0/wal/wals/c3934fb7b527497393e3cc4be252d3ca
I20260812 03:57:33.524797 2162 ts_tablet_manager.cc:1939] T c3934fb7b527497393e3cc4be252d3ca P 842f35c85ebb41dda94f9909e89de3a4: Deleting consensus metadata
I20260812 03:57:33.524852 2176 tablet_bootstrap.cc:492] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5: No bootstrap required, opened a new log
I20260812 03:57:33.524909 2176 ts_tablet_manager.cc:1397] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:33.525018 2176 raft_consensus.cc:348] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.525094 2176 raft_consensus.cc:374] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:33.525117 2176 raft_consensus.cc:729] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b09f63f9ed1349dfa00cfa58e68377c5, State: Initialized, Role: FOLLOWER
I20260812 03:57:33.525108 1434 catalog_manager.cc:4841] TS 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193:35569): tablet c3934fb7b527497393e3cc4be252d3ca (table test-table [id=f11521a2fc2f4cc4b1a6f7cfea8f28cd]) successfully deleted
I20260812 03:57:33.525167 2176 consensus_queue.cc:260] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.525211 2174 tablet_bootstrap.cc:492] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4: No bootstrap required, opened a new log
I20260812 03:57:33.525255 2174 ts_tablet_manager.cc:1397] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:33.525276 2176 ts_tablet_manager.cc:1428] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:57:33.525367 2174 raft_consensus.cc:348] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.525480 2174 raft_consensus.cc:374] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:33.525503 2174 raft_consensus.cc:729] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842f35c85ebb41dda94f9909e89de3a4, State: Initialized, Role: FOLLOWER
I20260812 03:57:33.525498 2160 ts_tablet_manager.cc:1918] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:33.525555 2160 log.cc:1196] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/c3934fb7b527497393e3cc4be252d3ca
I20260812 03:57:33.525553 2174 consensus_queue.cc:260] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.525655 2174 ts_tablet_manager.cc:1428] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:57:33.525790 2160 ts_tablet_manager.cc:1939] T c3934fb7b527497393e3cc4be252d3ca P a2a3fd59e9a54babb012b2f53e1da667: Deleting consensus metadata
I20260812 03:57:33.526240 1436 catalog_manager.cc:4841] TS a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951): tablet c3934fb7b527497393e3cc4be252d3ca (table test-table [id=f11521a2fc2f4cc4b1a6f7cfea8f28cd]) successfully deleted
I20260812 03:57:33.526381 2177 tablet_bootstrap.cc:492] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667: No bootstrap required, opened a new log
I20260812 03:57:33.526438 2177 ts_tablet_manager.cc:1397] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:57:33.526830 2177 raft_consensus.cc:348] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.526919 2177 raft_consensus.cc:374] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:33.526949 2177 raft_consensus.cc:729] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a3fd59e9a54babb012b2f53e1da667, State: Initialized, Role: FOLLOWER
I20260812 03:57:33.526997 2177 consensus_queue.cc:260] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.527091 2177 ts_tablet_manager.cc:1428] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:33.629185 2183 raft_consensus.cc:482] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:33.629307 2183 raft_consensus.cc:504] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.629503 2183 leader_election.cc:288] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194:35067), a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951)
I20260812 03:57:33.629781 1974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b768c98164c7a851aafb199d63f8e" candidate_uuid: "842f35c85ebb41dda94f9909e89de3a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a3fd59e9a54babb012b2f53e1da667" is_pre_election: true
I20260812 03:57:33.629788 1807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b768c98164c7a851aafb199d63f8e" candidate_uuid: "842f35c85ebb41dda94f9909e89de3a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" is_pre_election: true
I20260812 03:57:33.629899 1807 raft_consensus.cc:2454] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842f35c85ebb41dda94f9909e89de3a4 in term 0.
I20260812 03:57:33.629901 1974 raft_consensus.cc:2454] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842f35c85ebb41dda94f9909e89de3a4 in term 0.
I20260812 03:57:33.630119 1561 leader_election.cc:302] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842f35c85ebb41dda94f9909e89de3a4, a2a3fd59e9a54babb012b2f53e1da667; no voters:
I20260812 03:57:33.630237 2183 raft_consensus.cc:2781] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:33.630288 2183 raft_consensus.cc:482] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:33.630304 2183 raft_consensus.cc:3037] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:33.631217 2183 raft_consensus.cc:504] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.631383 2183 leader_election.cc:288] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [CANDIDATE]: Term 1 election: Requested vote from peers b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194:35067), a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951)
I20260812 03:57:33.631572 1974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b768c98164c7a851aafb199d63f8e" candidate_uuid: "842f35c85ebb41dda94f9909e89de3a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a3fd59e9a54babb012b2f53e1da667"
I20260812 03:57:33.631601 1807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b768c98164c7a851aafb199d63f8e" candidate_uuid: "842f35c85ebb41dda94f9909e89de3a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b09f63f9ed1349dfa00cfa58e68377c5"
I20260812 03:57:33.631677 1974 raft_consensus.cc:3037] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:33.631709 1807 raft_consensus.cc:3037] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:33.632440 1974 raft_consensus.cc:2454] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842f35c85ebb41dda94f9909e89de3a4 in term 1.
I20260812 03:57:33.632614 1807 raft_consensus.cc:2454] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842f35c85ebb41dda94f9909e89de3a4 in term 1.
I20260812 03:57:33.632663 1561 leader_election.cc:302] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842f35c85ebb41dda94f9909e89de3a4, a2a3fd59e9a54babb012b2f53e1da667; no voters:
I20260812 03:57:33.632748 2183 raft_consensus.cc:2781] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:33.632891 2183 raft_consensus.cc:686] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [term 1 LEADER]: Becoming Leader. State: Replica: 842f35c85ebb41dda94f9909e89de3a4, State: Running, Role: LEADER
I20260812 03:57:33.632973 2183 consensus_queue.cc:237] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } }
I20260812 03:57:33.633749 1455 catalog_manager.cc:5495] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193). New cstate: current_term: 1 leader_uuid: "842f35c85ebb41dda94f9909e89de3a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842f35c85ebb41dda94f9909e89de3a4" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 } health_report { overall_health: UNKNOWN } } }
I20260812 03:57:33.650269 2198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:33.646286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48172 (local address 127.31.43.193:35569)
0812 03:57:33.646515 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:33.646517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:33.646528 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:57:33.646560 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:33.646560 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:33.646575 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:33.646665 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:33.646850 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:33.647861 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:33.649161 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:33.649530 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:33.649621 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:33.649801 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:33.649867 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:33.649946 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:33.650081 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 03:57:33.650117 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":155,"threads_started":1}
I20260812 03:57:33.653697 1974 raft_consensus.cc:1261] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Refusing update from remote peer 842f35c85ebb41dda94f9909e89de3a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:33.653908 1807 raft_consensus.cc:1261] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Refusing update from remote peer 842f35c85ebb41dda94f9909e89de3a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:33.654044 2183 consensus_queue.cc:1035] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a3fd59e9a54babb012b2f53e1da667" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:33.654335 2183 consensus_queue.cc:1035] T ea2b768c98164c7a851aafb199d63f8e P 842f35c85ebb41dda94f9909e89de3a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b09f63f9ed1349dfa00cfa58e68377c5" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:57:33.660445 2142 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 03:57:35.185081 1454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:35.185202 1454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:35.186304 1454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8c0df41123c413eb57f172d1fceaa97: Sending DeleteTablet for 3 replicas of tablet ea2b768c98164c7a851aafb199d63f8e
I20260812 03:57:35.186633 1783 tablet_service.cc:1514] Processing DeleteTablet for tablet ea2b768c98164c7a851aafb199d63f8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:35 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:38492
I20260812 03:57:35.186718 1615 tablet_service.cc:1514] Processing DeleteTablet for tablet ea2b768c98164c7a851aafb199d63f8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:35 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:48144
I20260812 03:57:35.186802 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 1538
I20260812 03:57:35.189288 1948 tablet_service.cc:1514] Processing DeleteTablet for tablet ea2b768c98164c7a851aafb199d63f8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:35 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:53466
I20260812 03:57:35.189303 2231 tablet_replica.cc:330] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5: stopping tablet replica
I20260812 03:57:35.189379 2231 raft_consensus.cc:2229] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:35.189429 2231 raft_consensus.cc:2258] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:35.189608 2233 tablet_replica.cc:330] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667: stopping tablet replica
I20260812 03:57:35.189689 2233 raft_consensus.cc:2229] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:35.189736 2233 raft_consensus.cc:2258] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:35.189767 2231 ts_tablet_manager.cc:1905] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:35.190049 2233 ts_tablet_manager.cc:1905] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:35.191649 2231 ts_tablet_manager.cc:1918] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:57:35.191707 2231 log.cc:1196] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/ea2b768c98164c7a851aafb199d63f8e
I20260812 03:57:35.191937 2231 ts_tablet_manager.cc:1939] T ea2b768c98164c7a851aafb199d63f8e P b09f63f9ed1349dfa00cfa58e68377c5: Deleting consensus metadata
I20260812 03:57:35.192305 1438 catalog_manager.cc:4841] TS b09f63f9ed1349dfa00cfa58e68377c5 (127.31.43.194:35067): tablet ea2b768c98164c7a851aafb199d63f8e (table test-table [id=b693ce883e6a49f4b08004e6fa380ee1]) successfully deleted
I20260812 03:57:35.193405 2233 ts_tablet_manager.cc:1918] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:57:35.193480 2233 log.cc:1196] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/ea2b768c98164c7a851aafb199d63f8e
I20260812 03:57:35.193704 2233 ts_tablet_manager.cc:1939] T ea2b768c98164c7a851aafb199d63f8e P a2a3fd59e9a54babb012b2f53e1da667: Deleting consensus metadata
I20260812 03:57:35.194099 1436 catalog_manager.cc:4841] TS a2a3fd59e9a54babb012b2f53e1da667 (127.31.43.195:42951): tablet ea2b768c98164c7a851aafb199d63f8e (table test-table [id=b693ce883e6a49f4b08004e6fa380ee1]) successfully deleted
W20260812 03:57:35.199854 1434 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:35569 (error 108)
I20260812 03:57:35.200155 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 1709
I20260812 03:57:35.202425 2235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:35.200291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:35569 (local address 127.0.0.1:54046)
0812 03:57:35.202318 (+ 2027us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:35.202378 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:35569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1993,"thread_start_us":61,"threads_started":1}
W20260812 03:57:35.202584 1434 catalog_manager.cc:4584] TS 842f35c85ebb41dda94f9909e89de3a4 (127.31.43.193:35569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea2b768c98164c7a851aafb199d63f8e: Network error: Client connection negotiation failed: client connection to 127.31.43.193:35569: connect: Connection refused (error 111)
I20260812 03:57:35.210669 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 1874
I20260812 03:57:35.220495 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 1417
2026-08-12T03:57:35Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5823 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:35 dist-test-slave-8tdl krb5kdc[2238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:35 dist-test-slave-8tdl krb5kdc[2238](info): set up 2 sockets
Aug 12 03:57:35 dist-test-slave-8tdl krb5kdc[2238](info): commencing operation
krb5kdc: starting...
W20260812 03:57:36.353808 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:57:36 dist-test-slave-8tdl krb5kdc[2238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:57:36.495158 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:43127
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:43127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:36.587770 2252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:36.588081 2252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:36.588138 2252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:36.589530 2252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:36.589623 2252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:36.589651 2252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:36.589676 2252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:36.589709 2252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:36.591607 2252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:43127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:43127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:36.591954 2252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:36.592235 2252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:36.598071 2263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:36.598071 2261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:36.598140 2252 server_base.cc:1034] running on GCE node
W20260812 03:57:36.598050 2260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:36.598603 2252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:36.598968 2252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:36.600173 2252 hybrid_clock.cc:648] HybridClock initialized: now 1786507056600167 us; error 43 us; skew 500 ppm
Aug 12 03:57:36 dist-test-slave-8tdl krb5kdc[2238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507056, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:36.603245 2252 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:57:36.603749 2252 webserver.cc:460] Webserver started at http://127.31.43.254:41261/ using document root <none> and password file <none>
I20260812 03:57:36.604069 2252 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:36.604149 2252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:36.604314 2252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:36.605523 2252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "21be4aed23fe4fbf9d24419edacac99c"
format_stamp: "Formatted at 2026-08-12 03:57:36 on dist-test-slave-8tdl"
server_key: "7542a5a44b8d613d1d2b32d148cbedb0"
server_key_iv: "a9f1f50fb9d75b48c6fee6e58de5f793"
server_key_version: "encryptionkey@0"
I20260812 03:57:36.605952 2252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "21be4aed23fe4fbf9d24419edacac99c"
format_stamp: "Formatted at 2026-08-12 03:57:36 on dist-test-slave-8tdl"
server_key: "7542a5a44b8d613d1d2b32d148cbedb0"
server_key_iv: "a9f1f50fb9d75b48c6fee6e58de5f793"
server_key_version: "encryptionkey@0"
I20260812 03:57:36.607671 2252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 03:57:36.608618 2271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:36.608851 2252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:36.609042 2252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "21be4aed23fe4fbf9d24419edacac99c"
format_stamp: "Formatted at 2026-08-12 03:57:36 on dist-test-slave-8tdl"
server_key: "7542a5a44b8d613d1d2b32d148cbedb0"
server_key_iv: "a9f1f50fb9d75b48c6fee6e58de5f793"
server_key_version: "encryptionkey@0"
I20260812 03:57:36.609135 2252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:36.630193 2252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:36.632812 2252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:36.633013 2252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:36.637534 2252 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:43127
I20260812 03:57:36.637594 2333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:43127 every 8 connection(s)
I20260812 03:57:36.638000 2252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:36.639240 2334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:36.641817 2334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c: Bootstrap starting.
I20260812 03:57:36.641906 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 2252
I20260812 03:57:36.642063 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:36.642262 31919 external_mini_cluster.cc:1428] Setting key 5f688f8e61a74b17370118fb62e1c79a
I20260812 03:57:36.642668 2334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:36.643132 2334 log.cc:824] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:36.644014 2334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c: No bootstrap required, opened a new log
I20260812 03:57:36.646504 2334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21be4aed23fe4fbf9d24419edacac99c" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43127 } }
I20260812 03:57:36.646736 2334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:36.646785 2334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21be4aed23fe4fbf9d24419edacac99c, State: Initialized, Role: FOLLOWER
I20260812 03:57:36.646937 2334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21be4aed23fe4fbf9d24419edacac99c" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43127 } }
I20260812 03:57:36.647049 2334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:36.647095 2334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:36.647145 2334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:36.647949 2334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21be4aed23fe4fbf9d24419edacac99c" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43127 } }
I20260812 03:57:36.648097 2334 leader_election.cc:302] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21be4aed23fe4fbf9d24419edacac99c; no voters:
I20260812 03:57:36.648339 2334 leader_election.cc:288] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:36.648378 2339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:36.648490 2339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [term 1 LEADER]: Becoming Leader. State: Replica: 21be4aed23fe4fbf9d24419edacac99c, State: Running, Role: LEADER
I20260812 03:57:36.648658 2334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:36.648658 2339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21be4aed23fe4fbf9d24419edacac99c" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43127 } }
I20260812 03:57:36.649199 2341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21be4aed23fe4fbf9d24419edacac99c. Latest consensus state: current_term: 1 leader_uuid: "21be4aed23fe4fbf9d24419edacac99c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21be4aed23fe4fbf9d24419edacac99c" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43127 } } }
I20260812 03:57:36.649297 2341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:36.649820 2340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21be4aed23fe4fbf9d24419edacac99c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21be4aed23fe4fbf9d24419edacac99c" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43127 } } }
I20260812 03:57:36.649912 2340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c [sys.catalog]: This master's current role is: LEADER
Aug 12 03:57:36 dist-test-slave-8tdl krb5kdc[2238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:36.650357 2351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:36.650966 2351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:36.652808 2351 catalog_manager.cc:1347] Generated new cluster ID: af58e2639e5340a787dcab9ec93a07aa
I20260812 03:57:36.652859 2351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:36.654654 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:36.642889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54782 (local address 127.31.43.254:43127)
0812 03:57:36.643092 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:36.643100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:36.643127 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:57:36.643628 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:36.643635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:36.643688 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:36.643824 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:36.644688 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:36.645652 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:36.646808 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:36.647420 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:36.647514 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:36.650864 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:36.650869 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:36.650869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:36.650891 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:36.652932 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:36.653335 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:36.653339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:36.653339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:36.653436 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:36.653663 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:36.653664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:36.653664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:36.653986 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:36.654149 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:36.654272 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 03:57:36.654424 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260812 03:57:36.675060 2351 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:36.675707 2351 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:36.678933 2351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21be4aed23fe4fbf9d24419edacac99c: Generated new TSK 0
I20260812 03:57:36.679160 2351 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 03:57:36.715440 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43127
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:36.806466 2363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:36.806759 2363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:36.806792 2363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:36.808097 2363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:36.808148 2363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:36.808177 2363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:36.809726 2363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:36.810039 2363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:36.810312 2363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:36.816432 2369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:36.816432 2372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:36.816427 2370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:36.816722 2363 server_base.cc:1034] running on GCE node
I20260812 03:57:36.817050 2363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:36.817338 2363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:36.818552 2363 hybrid_clock.cc:648] HybridClock initialized: now 1786507056818501 us; error 67 us; skew 500 ppm
Aug 12 03:57:36 dist-test-slave-8tdl krb5kdc[2238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507056, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:36.821549 2363 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 03:57:36.822125 2363 webserver.cc:460] Webserver started at http://127.31.43.193:38727/ using document root <none> and password file <none>
I20260812 03:57:36.822400 2363 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:36.822475 2363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:36.822638 2363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:36.823801 2363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "15176074eb994bffbd5ee6821858e1e3"
format_stamp: "Formatted at 2026-08-12 03:57:36 on dist-test-slave-8tdl"
server_key: "9eaff2363ef577eae66e8dc1788ea851"
server_key_iv: "a57dfec8bb5350fcba974cbb0495d110"
server_key_version: "encryptionkey@0"
I20260812 03:57:36.824206 2363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "15176074eb994bffbd5ee6821858e1e3"
format_stamp: "Formatted at 2026-08-12 03:57:36 on dist-test-slave-8tdl"
server_key: "9eaff2363ef577eae66e8dc1788ea851"
server_key_iv: "a57dfec8bb5350fcba974cbb0495d110"
server_key_version: "encryptionkey@0"
I20260812 03:57:36.825657 2363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:36.826488 2384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:36.826762 2363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:36.826871 2363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "15176074eb994bffbd5ee6821858e1e3"
format_stamp: "Formatted at 2026-08-12 03:57:36 on dist-test-slave-8tdl"
server_key: "9eaff2363ef577eae66e8dc1788ea851"
server_key_iv: "a57dfec8bb5350fcba974cbb0495d110"
server_key_version: "encryptionkey@0"
I20260812 03:57:36.826960 2363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:36.838254 2363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:36.840920 2363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:36.841117 2363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:36.841392 2363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:36.841759 2363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:36.841816 2363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:36.841858 2363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:36.841902 2363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:36.848450 2363 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:33155
I20260812 03:57:36.848505 2515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:33155 every 8 connection(s)
I20260812 03:57:36.848862 2363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:57:36.851974 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 2363
I20260812 03:57:36.852090 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:57:36.852267 31919 external_mini_cluster.cc:1428] Setting key b485d81c14df5dc0cc44a7eb52a4827b
Aug 12 03:57:36 dist-test-slave-8tdl krb5kdc[2238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507056, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:36.859712 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:36.849659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:44461 (local address 127.31.43.254:43127)
0812 03:57:36.849739 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:36.849742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:36.850081 (+ 339us) server_negotiation.cc:407] Connection header received
0812 03:57:36.850601 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:36.850602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:36.850613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:36.850713 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:36.851752 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:36.852368 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:36.853769 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:36.854013 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:36.854080 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:36.856676 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:36.856678 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:36.856679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:36.856687 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:36.858159 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:36.858619 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:36.858620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:36.858620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:36.858722 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:36.858989 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:36.858990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:36.858990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:36.859120 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:36.859201 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:36.859498 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 03:57:36.859614 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:57:36.859920 2518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:36.849653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:43127 (local address 127.31.43.193:44461)
0812 03:57:36.849849 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:36.849914 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:36.850514 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:36.850809 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:36.850813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:36.851407 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:36.851640 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:36.851646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:36.852467 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:36.852468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:36.853676 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:36.853680 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:36.854113 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:36.854114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:36.854276 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:36.854979 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:36.854992 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:36.856567 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:36.858265 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:36.858274 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:36.858276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:36.858521 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:36.858779 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:36.858780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:36.858780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:36.858910 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:36.859191 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:36.859199 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:36.859352 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 03:57:36.859464 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 03:57:36.859648 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":108,"threads_started":1}
I20260812 03:57:36.860488 2516 heartbeater.cc:347] Connected to a master server at 127.31.43.254:43127
I20260812 03:57:36.860585 2516 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:36.860805 2516 heartbeater.cc:511] Master 127.31.43.254:43127 requested a full tablet report, sending...
I20260812 03:57:36.861423 2291 ts_manager.cc:194] Registered new tserver with Master: 15176074eb994bffbd5ee6821858e1e3 (127.31.43.193:33155)
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
I20260812 03:57:36.862125 2291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:44461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 03:57:36.883188 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43127
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:36.971504 2522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:36.971740 2522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:36.971768 2522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:36.973009 2522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:36.973057 2522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:36.973083 2522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:36.974576 2522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:36.974821 2522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:36.975065 2522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:36.980918 2528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:36.980918 2531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:36.981168 2522 server_base.cc:1034] running on GCE node
W20260812 03:57:36.980958 2527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:36.981495 2522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:36.981787 2522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:36.982935 2522 hybrid_clock.cc:648] HybridClock initialized: now 1786507056982922 us; error 34 us; skew 500 ppm
Aug 12 03:57:36 dist-test-slave-8tdl krb5kdc[2238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507056, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:36.985737 2522 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 03:57:36.986220 2522 webserver.cc:460] Webserver started at http://127.31.43.194:40257/ using document root <none> and password file <none>
I20260812 03:57:36.986441 2522 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:36.986485 2522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:36.986580 2522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:36.987746 2522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "62810616e311445ea0eca05ced4458aa"
format_stamp: "Formatted at 2026-08-12 03:57:36 on dist-test-slave-8tdl"
server_key: "e34a12a8ab2bc762b8b2ba074ab45c2e"
server_key_iv: "8941c5e63186844a818b6c0b7c3a3a1f"
server_key_version: "encryptionkey@0"
I20260812 03:57:36.988137 2522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "62810616e311445ea0eca05ced4458aa"
format_stamp: "Formatted at 2026-08-12 03:57:36 on dist-test-slave-8tdl"
server_key: "e34a12a8ab2bc762b8b2ba074ab45c2e"
server_key_iv: "8941c5e63186844a818b6c0b7c3a3a1f"
server_key_version: "encryptionkey@0"
I20260812 03:57:36.989583 2522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:57:36.990433 2540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:36.990648 2522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:36.990792 2522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "62810616e311445ea0eca05ced4458aa"
format_stamp: "Formatted at 2026-08-12 03:57:36 on dist-test-slave-8tdl"
server_key: "e34a12a8ab2bc762b8b2ba074ab45c2e"
server_key_iv: "8941c5e63186844a818b6c0b7c3a3a1f"
server_key_version: "encryptionkey@0"
I20260812 03:57:36.990877 2522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:37.007690 2522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:37.010334 2522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:37.010520 2522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:37.010882 2522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:37.011227 2522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:37.011283 2522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:37.011327 2522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:37.011354 2522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:37.017819 2522 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:38477
I20260812 03:57:37.017875 2671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:38477 every 8 connection(s)
I20260812 03:57:37.018231 2522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:57:37.019797 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 2522
I20260812 03:57:37.019887 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:57:37.020159 31919 external_mini_cluster.cc:1428] Setting key c96038828101ed489298902d609e7604
Aug 12 03:57:37 dist-test-slave-8tdl krb5kdc[2238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507056, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:37.029430 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:37.019111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:58899 (local address 127.31.43.254:43127)
0812 03:57:37.019169 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:37.019172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:37.019380 (+ 208us) server_negotiation.cc:407] Connection header received
0812 03:57:37.019991 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:37.019993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:37.020008 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:37.020071 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:37.021384 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:37.021891 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:37.023369 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:37.023591 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:37.023659 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:37.026256 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:37.026257 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:37.026258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:37.026272 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:37.027867 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:37.028384 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:37.028386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:37.028386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:37.028452 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:37.028719 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:37.028721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:37.028721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:37.028846 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:37.028921 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:37.029228 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 03:57:37.029334 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:57:37.029651 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:37.019042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:43127 (local address 127.31.43.194:58899)
0812 03:57:37.019221 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:37.019274 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:37.019883 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:37.020164 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:37.020173 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:37.021052 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:37.021282 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:37.021288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:37.022006 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:37.022007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:37.023252 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:37.023257 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:37.023688 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:37.023690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:37.023859 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:37.024431 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:37.024441 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:37.026135 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:37.028043 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:37.028052 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:37.028055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:37.028302 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:37.028530 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:37.028531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:37.028532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:37.028652 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:37.028925 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:37.028933 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:37.029084 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 03:57:37.029193 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 03:57:37.029375 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
I20260812 03:57:37.030212 2672 heartbeater.cc:347] Connected to a master server at 127.31.43.254:43127
I20260812 03:57:37.030323 2672 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:37.030516 2672 heartbeater.cc:511] Master 127.31.43.254:43127 requested a full tablet report, sending...
I20260812 03:57:37.030977 2291 ts_manager.cc:194] Registered new tserver with Master: 62810616e311445ea0eca05ced4458aa (127.31.43.194:38477)
I20260812 03:57:37.031414 2291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:58899
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 03:57:37.051213 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43127
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:37.138947 2677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:37.139189 2677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:37.139215 2677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:37.140586 2677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:37.140641 2677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:37.140671 2677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:37.142274 2677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:37.142544 2677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:37.142817 2677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:37.148329 2686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:37.148326 2683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:37.148551 2677 server_base.cc:1034] running on GCE node
W20260812 03:57:37.148598 2684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:37.148852 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:37.149211 2677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:37.150437 2677 hybrid_clock.cc:648] HybridClock initialized: now 1786507057150434 us; error 51 us; skew 500 ppm
Aug 12 03:57:37 dist-test-slave-8tdl krb5kdc[2238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507057, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:37.153309 2677 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 03:57:37.153887 2677 webserver.cc:460] Webserver started at http://127.31.43.195:35653/ using document root <none> and password file <none>
I20260812 03:57:37.154172 2677 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:37.154246 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:37.154408 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:37.155609 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "a69cc56e38ae4c7cb91b4891083c61e4"
format_stamp: "Formatted at 2026-08-12 03:57:37 on dist-test-slave-8tdl"
server_key: "6e98ece49c94a637515bf5411c699218"
server_key_iv: "0de377ae97877173293e22f043025033"
server_key_version: "encryptionkey@0"
I20260812 03:57:37.156004 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "a69cc56e38ae4c7cb91b4891083c61e4"
format_stamp: "Formatted at 2026-08-12 03:57:37 on dist-test-slave-8tdl"
server_key: "6e98ece49c94a637515bf5411c699218"
server_key_iv: "0de377ae97877173293e22f043025033"
server_key_version: "encryptionkey@0"
I20260812 03:57:37.157634 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:57:37.158457 2694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:37.158644 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:37.158768 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "a69cc56e38ae4c7cb91b4891083c61e4"
format_stamp: "Formatted at 2026-08-12 03:57:37 on dist-test-slave-8tdl"
server_key: "6e98ece49c94a637515bf5411c699218"
server_key_iv: "0de377ae97877173293e22f043025033"
server_key_version: "encryptionkey@0"
I20260812 03:57:37.158852 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:37.174074 2677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:37.176816 2677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:37.177017 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:37.177305 2677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:37.177662 2677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:37.177721 2677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:37.177763 2677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:37.177793 2677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:37.184721 2677 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:40541
I20260812 03:57:37.184777 2825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:40541 every 8 connection(s)
I20260812 03:57:37.185176 2677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:57:37.187886 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 2677
I20260812 03:57:37.187992 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:57:37.188198 31919 external_mini_cluster.cc:1428] Setting key 44b2c6ceb6be8c1d7b71df6b3643b832
Aug 12 03:57:37 dist-test-slave-8tdl krb5kdc[2238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507057, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:37.195915 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:37.186154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:43165 (local address 127.31.43.254:43127)
0812 03:57:37.186221 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:37.186223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:37.186558 (+ 335us) server_negotiation.cc:407] Connection header received
0812 03:57:37.187033 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:37.187034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:37.187047 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:37.187125 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:37.188470 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:37.189200 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:37.190344 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:37.190598 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:37.190669 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:37.193007 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:37.193009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:37.193009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:37.193016 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:37.194523 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:37.195086 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:37.195087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:37.195087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:37.195142 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:37.195372 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:37.195373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:37.195373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:37.195482 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:37.195545 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:37.195779 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 03:57:37.195846 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:57:37.196123 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:37.186113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:43127 (local address 127.31.43.195:43165)
0812 03:57:37.186292 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:37.186339 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:37.186921 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:37.187191 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:37.187195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:37.187938 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:37.188264 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:37.188272 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:37.189299 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:37.189301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:37.190238 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:37.190241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:37.190745 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:37.190746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:37.190872 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:37.191403 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:37.191412 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:37.192909 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:37.194616 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:37.194623 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:37.194625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:37.195012 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:37.195204 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:37.195205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:37.195205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:37.195311 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:37.195534 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:37.195538 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:37.195658 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 03:57:37.195738 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 03:57:37.195868 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260812 03:57:37.196815 2826 heartbeater.cc:347] Connected to a master server at 127.31.43.254:43127
I20260812 03:57:37.196939 2826 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:37.197142 2826 heartbeater.cc:511] Master 127.31.43.254:43127 requested a full tablet report, sending...
I20260812 03:57:37.197578 2291 ts_manager.cc:194] Registered new tserver with Master: a69cc56e38ae4c7cb91b4891083c61e4 (127.31.43.195:40541)
I20260812 03:57:37.197973 2291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:43165
I20260812 03:57:37.200345 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:57:37.214800 2337 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:37.213629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54798 (local address 127.31.43.254:43127)
0812 03:57:37.213712 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:37.213714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:37.213878 (+ 164us) server_negotiation.cc:407] Connection header received
0812 03:57:37.213927 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:37.213928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:37.213938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:37.213997 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:37.214737 (+ 740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54798: BlockingRecv error: recv got EOF from 127.0.0.1:54798 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:57:37.215971 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 2363
I20260812 03:57:37.227077 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 2522
I20260812 03:57:37.234382 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 2677
I20260812 03:57:37.240777 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 2252
2026-08-12T03:57:37Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2018 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:37 dist-test-slave-8tdl krb5kdc[2840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:37 dist-test-slave-8tdl krb5kdc[2840](info): set up 2 sockets
Aug 12 03:57:37 dist-test-slave-8tdl krb5kdc[2840](info): commencing operation
krb5kdc: starting...
W20260812 03:57:38.369287 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:57:38 dist-test-slave-8tdl krb5kdc[2840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:57:38.515427 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:45523
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:45523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:38.608415 2856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:38.608703 2856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:38.608747 2856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:38.610008 2856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:38.610088 2856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:38.610113 2856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:38.610136 2856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:38.610162 2856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:38.611927 2856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:45523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:45523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:38.612241 2856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:38.612511 2856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:38.617924 2862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:38.618135 2863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:38.617940 2865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:38.618459 2856 server_base.cc:1034] running on GCE node
I20260812 03:57:38.618775 2856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:38.619196 2856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:38.620378 2856 hybrid_clock.cc:648] HybridClock initialized: now 1786507058620362 us; error 52 us; skew 500 ppm
Aug 12 03:57:38 dist-test-slave-8tdl krb5kdc[2840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507058, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:38.627447 2856 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:57:38.628249 2856 webserver.cc:460] Webserver started at http://127.31.43.254:38821/ using document root <none> and password file <none>
I20260812 03:57:38.628575 2856 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:38.628634 2856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:38.628827 2856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:38.630122 2856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "e64e9eb6e31e44198dff12f9fd709482"
format_stamp: "Formatted at 2026-08-12 03:57:38 on dist-test-slave-8tdl"
server_key: "a07ee282269f6b2bb60e3b4709c572fc"
server_key_iv: "dc0934e83cb15e151d8323bfab52c075"
server_key_version: "encryptionkey@0"
I20260812 03:57:38.630630 2856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "e64e9eb6e31e44198dff12f9fd709482"
format_stamp: "Formatted at 2026-08-12 03:57:38 on dist-test-slave-8tdl"
server_key: "a07ee282269f6b2bb60e3b4709c572fc"
server_key_iv: "dc0934e83cb15e151d8323bfab52c075"
server_key_version: "encryptionkey@0"
I20260812 03:57:38.632454 2856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:38.633481 2872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:38.633733 2856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:38.633859 2856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "e64e9eb6e31e44198dff12f9fd709482"
format_stamp: "Formatted at 2026-08-12 03:57:38 on dist-test-slave-8tdl"
server_key: "a07ee282269f6b2bb60e3b4709c572fc"
server_key_iv: "dc0934e83cb15e151d8323bfab52c075"
server_key_version: "encryptionkey@0"
I20260812 03:57:38.633991 2856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:38.645999 2856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:38.648625 2856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:38.648851 2856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:38.654124 2856 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:45523
I20260812 03:57:38.654174 2930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:45523 every 8 connection(s)
I20260812 03:57:38.654824 2856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:38.655627 2931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:38.658273 2931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482: Bootstrap starting.
I20260812 03:57:38.659119 2931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:38.659466 2931 log.cc:824] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:38.660485 2931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482: No bootstrap required, opened a new log
I20260812 03:57:38.662317 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 2856
I20260812 03:57:38.662434 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:38.662628 31919 external_mini_cluster.cc:1428] Setting key 8a54c8a80cb541019c24116d23ef58d6
I20260812 03:57:38.662779 2931 raft_consensus.cc:348] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64e9eb6e31e44198dff12f9fd709482" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 45523 } }
I20260812 03:57:38.662989 2931 raft_consensus.cc:374] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:38.663072 2931 raft_consensus.cc:729] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64e9eb6e31e44198dff12f9fd709482, State: Initialized, Role: FOLLOWER
I20260812 03:57:38.663228 2931 consensus_queue.cc:260] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64e9eb6e31e44198dff12f9fd709482" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 45523 } }
I20260812 03:57:38.663337 2931 raft_consensus.cc:388] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:38.663398 2931 raft_consensus.cc:482] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:38.663456 2931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:38.664220 2931 raft_consensus.cc:504] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64e9eb6e31e44198dff12f9fd709482" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 45523 } }
I20260812 03:57:38.664338 2931 leader_election.cc:302] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e64e9eb6e31e44198dff12f9fd709482; no voters:
I20260812 03:57:38.664512 2931 leader_election.cc:288] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:38.664570 2936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:38.664801 2936 raft_consensus.cc:686] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [term 1 LEADER]: Becoming Leader. State: Replica: e64e9eb6e31e44198dff12f9fd709482, State: Running, Role: LEADER
I20260812 03:57:38.664930 2931 sys_catalog.cc:564] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:38.664995 2936 consensus_queue.cc:237] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64e9eb6e31e44198dff12f9fd709482" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 45523 } }
I20260812 03:57:38.665491 2936 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e64e9eb6e31e44198dff12f9fd709482. Latest consensus state: current_term: 1 leader_uuid: "e64e9eb6e31e44198dff12f9fd709482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64e9eb6e31e44198dff12f9fd709482" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 45523 } } }
I20260812 03:57:38.665601 2936 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:38.665508 2937 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e64e9eb6e31e44198dff12f9fd709482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64e9eb6e31e44198dff12f9fd709482" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 45523 } } }
I20260812 03:57:38.665789 2937 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482 [sys.catalog]: This master's current role is: LEADER
W20260812 03:57:38.667302 2954 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:57:38.667428 2954 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:57:38.667526 2945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:38.667696 2945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:38.669400 2945 catalog_manager.cc:1347] Generated new cluster ID: fabfff76d7494dffb1ef8b9c3bef57cd
I20260812 03:57:38.669488 2945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:57:38 dist-test-slave-8tdl krb5kdc[2840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:38.675982 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:38.663342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42880 (local address 127.31.43.254:45523)
0812 03:57:38.664019 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:38.664030 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:38.664070 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:57:38.664204 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:38.664212 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:38.664279 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:38.664459 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:38.665741 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:38.667324 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:38.668316 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:38.668942 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:38.669072 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:38.672036 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:38.672041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:38.672042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:38.672065 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:38.674022 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:38.674478 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:38.674482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:38.674482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:38.674590 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:38.674885 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:38.674886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:38.674886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:38.675177 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:38.675402 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:38.675484 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 03:57:38.675769 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":549,"threads_started":1}
I20260812 03:57:38.677047 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:57:38.691425 2945 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:38.692315 2945 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:38.695221 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:38.690652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42896 (local address 127.31.43.254:45523)
0812 03:57:38.690732 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:38.690735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:38.690971 (+ 236us) server_negotiation.cc:407] Connection header received
0812 03:57:38.691260 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:38.691261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:38.691276 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:38.691312 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:38.691889 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:38.692464 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:38.693339 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:38.693575 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:38.693665 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:38.694535 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:38.694536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:38.694536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:38.694856 (+ 320us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:38.694897 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:38.694905 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:38.694980 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:38.695056 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:57:38.695110 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:57:38.701663 2945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e64e9eb6e31e44198dff12f9fd709482: Generated new TSK 0
I20260812 03:57:38.701908 2945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:57:38.708065 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 2856
2026-08-12T03:57:38Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1465 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:38 dist-test-slave-8tdl krb5kdc[2968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:38 dist-test-slave-8tdl krb5kdc[2968](info): set up 2 sockets
Aug 12 03:57:38 dist-test-slave-8tdl krb5kdc[2968](info): commencing operation
krb5kdc: starting...
W20260812 03:57:39.836516 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:57:39 dist-test-slave-8tdl krb5kdc[2968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:57:39.982510 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:34489
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:34489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:40.072727 2983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:40.073009 2983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:40.073053 2983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:40.074570 2983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:40.074679 2983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:40.074734 2983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:40.074792 2983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:40.074823 2983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:40.076581 2983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:34489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:34489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:40.076939 2983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:40.077229 2983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:40.083357 2992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:40.083273 2990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:40.083479 2983 server_base.cc:1034] running on GCE node
W20260812 03:57:40.083284 2989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:40.083871 2983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:40.084239 2983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:40.085446 2983 hybrid_clock.cc:648] HybridClock initialized: now 1786507060085419 us; error 41 us; skew 500 ppm
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[2968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507060, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:40.088788 2983 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:57:40.089310 2983 webserver.cc:460] Webserver started at http://127.31.43.254:35385/ using document root <none> and password file <none>
I20260812 03:57:40.089577 2983 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:40.089653 2983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:40.089807 2983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:40.091167 2983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "3970846804294947a5ed0f287afa4156"
format_stamp: "Formatted at 2026-08-12 03:57:40 on dist-test-slave-8tdl"
server_key: "629274ed0b36314f0c520a4fa560bdb3"
server_key_iv: "efcfcd8e8f14aa3561b9a87ffabafc27"
server_key_version: "encryptionkey@0"
I20260812 03:57:40.091579 2983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "3970846804294947a5ed0f287afa4156"
format_stamp: "Formatted at 2026-08-12 03:57:40 on dist-test-slave-8tdl"
server_key: "629274ed0b36314f0c520a4fa560bdb3"
server_key_iv: "efcfcd8e8f14aa3561b9a87ffabafc27"
server_key_version: "encryptionkey@0"
I20260812 03:57:40.093309 2983 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:57:40.094393 3001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:40.094734 2983 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:40.094887 2983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "3970846804294947a5ed0f287afa4156"
format_stamp: "Formatted at 2026-08-12 03:57:40 on dist-test-slave-8tdl"
server_key: "629274ed0b36314f0c520a4fa560bdb3"
server_key_iv: "efcfcd8e8f14aa3561b9a87ffabafc27"
server_key_version: "encryptionkey@0"
I20260812 03:57:40.094985 2983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:40.108770 2983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:40.111328 2983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:40.111536 2983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:40.116555 2983 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:34489
I20260812 03:57:40.116587 3061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:34489 every 8 connection(s)
I20260812 03:57:40.117002 2983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:40.118374 3062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:40.119532 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 2983
I20260812 03:57:40.119665 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:40.119858 31919 external_mini_cluster.cc:1428] Setting key 48b85ec7211c1b65267820658f4a9799
I20260812 03:57:40.122733 3062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156: Bootstrap starting.
I20260812 03:57:40.123553 3062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:40.123927 3062 log.cc:824] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:40.124974 3062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156: No bootstrap required, opened a new log
I20260812 03:57:40.127858 3062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3970846804294947a5ed0f287afa4156" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34489 } }
I20260812 03:57:40.128055 3062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:40.128082 3062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3970846804294947a5ed0f287afa4156, State: Initialized, Role: FOLLOWER
I20260812 03:57:40.128222 3062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3970846804294947a5ed0f287afa4156" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34489 } }
I20260812 03:57:40.128314 3062 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:40.128389 3062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:40.128446 3062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[2968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:40.129431 3062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3970846804294947a5ed0f287afa4156" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34489 } }
I20260812 03:57:40.129582 3062 leader_election.cc:302] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3970846804294947a5ed0f287afa4156; no voters:
I20260812 03:57:40.129827 3062 leader_election.cc:288] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:40.129860 3068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:40.129997 3068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [term 1 LEADER]: Becoming Leader. State: Replica: 3970846804294947a5ed0f287afa4156, State: Running, Role: LEADER
I20260812 03:57:40.130119 3068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3970846804294947a5ed0f287afa4156" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34489 } }
I20260812 03:57:40.130165 3062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:40.130645 3070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3970846804294947a5ed0f287afa4156" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3970846804294947a5ed0f287afa4156" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34489 } } }
I20260812 03:57:40.130826 3070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:40.130795 3071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3970846804294947a5ed0f287afa4156. Latest consensus state: current_term: 1 leader_uuid: "3970846804294947a5ed0f287afa4156" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3970846804294947a5ed0f287afa4156" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34489 } } }
I20260812 03:57:40.130961 3071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:40.131307 3076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:40.132375 3076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:40.134084 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:40.120612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.31.43.254:34489)
0812 03:57:40.120847 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:40.120856 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:40.120885 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:57:40.121416 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:40.121430 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:40.121486 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:40.121639 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:40.122745 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.123914 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.125219 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.125665 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.125760 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:40.129230 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:40.129234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:40.129235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:40.129249 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:40.131869 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:40.132353 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:40.132357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:40.132357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:40.132496 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:40.132778 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:40.132779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:40.132780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:40.133056 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:40.133228 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:40.133701 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 03:57:40.133877 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":135,"threads_started":1}
I20260812 03:57:40.134444 3076 catalog_manager.cc:1347] Generated new cluster ID: 66cb8859f3a743679b57d1dcbef8bb87
I20260812 03:57:40.134505 3076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:40.171842 3076 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:40.172757 3076 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:40.177788 3076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3970846804294947a5ed0f287afa4156: Generated new TSK 0
I20260812 03:57:40.178073 3076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 03:57:40.219131 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:34489
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:40.313006 3091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:40.313323 3091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:40.313374 3091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:40.314994 3091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:40.315096 3091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:40.315152 3091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:40.317078 3091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:34489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:40.317436 3091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:40.317723 3091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:40.324290 3096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:40.324290 3100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:40.324644 3091 server_base.cc:1034] running on GCE node
W20260812 03:57:40.324493 3097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:40.325146 3091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:40.325527 3091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:40.326774 3091 hybrid_clock.cc:648] HybridClock initialized: now 1786507060326732 us; error 56 us; skew 500 ppm
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[2968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507060, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:40.330119 3091 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 03:57:40.330897 3091 webserver.cc:460] Webserver started at http://127.31.43.193:33535/ using document root <none> and password file <none>
I20260812 03:57:40.331171 3091 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:40.331226 3091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:40.331393 3091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:40.332669 3091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "3de6503ace824ee181dac7d5767aa923"
format_stamp: "Formatted at 2026-08-12 03:57:40 on dist-test-slave-8tdl"
server_key: "2c47bc683ad749ad8d43d4b60e658c63"
server_key_iv: "bd2368fcc8d7dbff4fc6adc93de9e27f"
server_key_version: "encryptionkey@0"
I20260812 03:57:40.333070 3091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "3de6503ace824ee181dac7d5767aa923"
format_stamp: "Formatted at 2026-08-12 03:57:40 on dist-test-slave-8tdl"
server_key: "2c47bc683ad749ad8d43d4b60e658c63"
server_key_iv: "bd2368fcc8d7dbff4fc6adc93de9e27f"
server_key_version: "encryptionkey@0"
I20260812 03:57:40.335143 3091 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 03:57:40.336249 3107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:40.336561 3091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:40.336701 3091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "3de6503ace824ee181dac7d5767aa923"
format_stamp: "Formatted at 2026-08-12 03:57:40 on dist-test-slave-8tdl"
server_key: "2c47bc683ad749ad8d43d4b60e658c63"
server_key_iv: "bd2368fcc8d7dbff4fc6adc93de9e27f"
server_key_version: "encryptionkey@0"
I20260812 03:57:40.336814 3091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:40.351073 3091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:40.353523 3091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:40.353722 3091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:40.354004 3091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:40.354380 3091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:40.354449 3091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:40.354486 3091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:40.354534 3091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:40.361203 3091 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:34185
I20260812 03:57:40.361372 3241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:34185 every 8 connection(s)
I20260812 03:57:40.361686 3091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:57:40.366554 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 3091
I20260812 03:57:40.366746 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:57:40.366991 31919 external_mini_cluster.cc:1428] Setting key 066d964210fd6387a769fe9c244fa649
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[2968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507060, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:40.372644 3244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:40.362510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:34489 (local address 127.31.43.193:54023)
0812 03:57:40.362711 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:40.362765 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:40.363315 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:40.363612 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:40.363617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:40.364350 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:40.364584 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:40.364589 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.365342 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.365343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:40.366187 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:40.366190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.366632 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.366633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:40.366838 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:40.367460 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:40.367470 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:40.369095 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:40.370999 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:40.371010 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:40.371013 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:40.371337 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:40.371616 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:40.371617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:40.371617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:40.371747 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:40.372058 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:40.372064 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:40.372200 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 03:57:40.372293 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 03:57:40.372464 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
I20260812 03:57:40.372623 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:40.362565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:54023 (local address 127.31.43.254:34489)
0812 03:57:40.362614 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:40.362617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:40.362852 (+ 235us) server_negotiation.cc:407] Connection header received
0812 03:57:40.363457 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:40.363458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:40.363471 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:40.363524 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:40.364711 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.365225 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.366289 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.366523 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.366618 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:40.369192 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:40.369194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:40.369195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:40.369204 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:40.370888 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:40.371435 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:40.371436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:40.371437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:40.371521 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:40.371820 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:40.371821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:40.371822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:40.371979 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:40.372056 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:40.372355 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 03:57:40.372476 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:57:40.373353 3242 heartbeater.cc:347] Connected to a master server at 127.31.43.254:34489
I20260812 03:57:40.373498 3242 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:40.373714 3242 heartbeater.cc:511] Master 127.31.43.254:34489 requested a full tablet report, sending...
I20260812 03:57:40.374396 3022 ts_manager.cc:194] Registered new tserver with Master: 3de6503ace824ee181dac7d5767aa923 (127.31.43.193:34185)
I20260812 03:57:40.375489 3022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:54023
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 03:57:40.404132 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:34489
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:40.496315 3248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:40.496572 3248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:40.496600 3248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:40.497907 3248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:40.497992 3248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:40.498081 3248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:40.499872 3248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:34489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:40.500176 3248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:40.500480 3248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:57:40.507651 3248 server_base.cc:1034] running on GCE node
W20260812 03:57:40.507588 3255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:40.507588 3258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:40.507630 3256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:40.508204 3248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:40.508527 3248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:40.509768 3248 hybrid_clock.cc:648] HybridClock initialized: now 1786507060509739 us; error 54 us; skew 500 ppm
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[2968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507060, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:40.512777 3248 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 03:57:40.513427 3248 webserver.cc:460] Webserver started at http://127.31.43.194:36277/ using document root <none> and password file <none>
I20260812 03:57:40.513713 3248 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:40.513792 3248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:40.513959 3248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:40.515237 3248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "9612012a44f5421893b15b311bfaa123"
format_stamp: "Formatted at 2026-08-12 03:57:40 on dist-test-slave-8tdl"
server_key: "758090a30f319b2e83e60aaebd418968"
server_key_iv: "2915175fefc4df770a71d5c0cc0b1ed4"
server_key_version: "encryptionkey@0"
I20260812 03:57:40.515641 3248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "9612012a44f5421893b15b311bfaa123"
format_stamp: "Formatted at 2026-08-12 03:57:40 on dist-test-slave-8tdl"
server_key: "758090a30f319b2e83e60aaebd418968"
server_key_iv: "2915175fefc4df770a71d5c0cc0b1ed4"
server_key_version: "encryptionkey@0"
I20260812 03:57:40.517570 3248 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:57:40.518476 3265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:40.518754 3248 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:40.518873 3248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "9612012a44f5421893b15b311bfaa123"
format_stamp: "Formatted at 2026-08-12 03:57:40 on dist-test-slave-8tdl"
server_key: "758090a30f319b2e83e60aaebd418968"
server_key_iv: "2915175fefc4df770a71d5c0cc0b1ed4"
server_key_version: "encryptionkey@0"
I20260812 03:57:40.519073 3248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:40.534067 3248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:40.536752 3248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:40.536970 3248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:40.537256 3248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:40.537624 3248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:40.537681 3248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:40.537724 3248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:40.537755 3248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:40.544713 3248 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:37355
I20260812 03:57:40.544760 3401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:37355 every 8 connection(s)
I20260812 03:57:40.545135 3248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:57:40.551239 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 3248
I20260812 03:57:40.551397 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:57:40.551626 31919 external_mini_cluster.cc:1428] Setting key 5faaba89251bb104a9cc2084976ba342
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[2968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507060, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:40.557307 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:40.545902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:44341 (local address 127.31.43.254:34489)
0812 03:57:40.545976 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:40.545978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:40.546159 (+ 181us) server_negotiation.cc:407] Connection header received
0812 03:57:40.546728 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:40.546730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:40.546743 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:40.546839 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:40.547925 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.548498 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.550173 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.550392 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.550462 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:40.553672 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:40.553674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:40.553674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:40.553692 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:40.555518 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:40.556097 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:40.556099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:40.556100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:40.556177 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:40.556472 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:40.556473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:40.556473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:40.556646 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:40.556722 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:40.557030 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 03:57:40.557150 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:57:40.557444 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:40.545813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:34489 (local address 127.31.43.194:44341)
0812 03:57:40.546017 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:40.546064 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:40.546588 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:40.546932 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:40.546936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:40.547578 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:40.547798 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:40.547803 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.548678 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.548679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:40.550057 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:40.550061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.550499 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.550500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:40.550618 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:40.551567 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:40.551579 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:40.553508 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:40.555632 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:40.555644 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:40.555647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:40.555990 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:40.556260 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:40.556261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:40.556261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:40.556398 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:40.556718 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:40.556723 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:40.556867 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 03:57:40.556962 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 03:57:40.557181 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260812 03:57:40.558163 3402 heartbeater.cc:347] Connected to a master server at 127.31.43.254:34489
I20260812 03:57:40.558303 3402 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:40.558542 3402 heartbeater.cc:511] Master 127.31.43.254:34489 requested a full tablet report, sending...
I20260812 03:57:40.559100 3022 ts_manager.cc:194] Registered new tserver with Master: 9612012a44f5421893b15b311bfaa123 (127.31.43.194:37355)
I20260812 03:57:40.559700 3022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:44341
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 03:57:40.586414 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:34489
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:40.677542 3407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:40.677870 3407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:40.677904 3407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:40.679309 3407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:40.679358 3407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:40.679386 3407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:40.680929 3407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:34489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:40.681212 3407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:40.681469 3407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:40.687253 3413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:40.687284 3414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:40.687258 3416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:40.687431 3407 server_base.cc:1034] running on GCE node
I20260812 03:57:40.687882 3407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:40.688144 3407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:40.689309 3407 hybrid_clock.cc:648] HybridClock initialized: now 1786507060689265 us; error 60 us; skew 500 ppm
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[2968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507060, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:40.692828 3407 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 03:57:40.693331 3407 webserver.cc:460] Webserver started at http://127.31.43.195:41009/ using document root <none> and password file <none>
I20260812 03:57:40.693580 3407 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:40.693652 3407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:40.693812 3407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:40.695039 3407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "cb9ef779b4b8470583105f42a901cf3c"
format_stamp: "Formatted at 2026-08-12 03:57:40 on dist-test-slave-8tdl"
server_key: "2aac3b27a34620142f29d173a8dd9814"
server_key_iv: "302823a43858862ddbc5f3c9fbbf3267"
server_key_version: "encryptionkey@0"
I20260812 03:57:40.695418 3407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "cb9ef779b4b8470583105f42a901cf3c"
format_stamp: "Formatted at 2026-08-12 03:57:40 on dist-test-slave-8tdl"
server_key: "2aac3b27a34620142f29d173a8dd9814"
server_key_iv: "302823a43858862ddbc5f3c9fbbf3267"
server_key_version: "encryptionkey@0"
I20260812 03:57:40.696853 3407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:57:40.697799 3423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:40.698030 3407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:40.698146 3407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "cb9ef779b4b8470583105f42a901cf3c"
format_stamp: "Formatted at 2026-08-12 03:57:40 on dist-test-slave-8tdl"
server_key: "2aac3b27a34620142f29d173a8dd9814"
server_key_iv: "302823a43858862ddbc5f3c9fbbf3267"
server_key_version: "encryptionkey@0"
I20260812 03:57:40.698225 3407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:40.711297 3407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:40.713914 3407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:40.714109 3407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:40.714375 3407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:40.714776 3407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:40.714830 3407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:40.714887 3407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:40.714917 3407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:40.721622 3407 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:38489
I20260812 03:57:40.721710 3547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:38489 every 8 connection(s)
I20260812 03:57:40.722121 3407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:57:40.723268 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 3407
I20260812 03:57:40.723380 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:57:40.723562 31919 external_mini_cluster.cc:1428] Setting key 0086110d896c0a3e0503fb5982f7b23e
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[2968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507060, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:40.733119 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:40.723063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:54265 (local address 127.31.43.254:34489)
0812 03:57:40.723190 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:40.723192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:40.723361 (+ 169us) server_negotiation.cc:407] Connection header received
0812 03:57:40.724137 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:40.724138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:40.724151 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:40.724220 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:40.725342 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.725872 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.727036 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.727247 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.727309 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:40.729698 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:40.729700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:40.729701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:40.729717 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:40.731541 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:40.732133 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:40.732134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:40.732135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:40.732191 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:40.732491 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:40.732491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:40.732492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:40.732607 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:40.732660 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:40.732930 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 03:57:40.733000 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 03:57:40.733362 3550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:40.723038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:34489 (local address 127.31.43.195:54265)
0812 03:57:40.723207 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:40.723252 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:40.724024 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:40.724298 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:40.724302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:40.725004 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:40.725231 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:40.725237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.726033 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.726034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:40.726954 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:40.726957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.727330 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.727331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:40.727444 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:40.728027 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:40.728038 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:40.729588 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:40.731669 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:40.731680 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:40.731683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:40.732060 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:40.732278 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:40.732279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:40.732279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:40.732422 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:40.732670 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:40.732675 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:40.732808 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 03:57:40.732910 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 03:57:40.733081 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260812 03:57:40.733959 3548 heartbeater.cc:347] Connected to a master server at 127.31.43.254:34489
I20260812 03:57:40.734079 3548 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:40.734287 3548 heartbeater.cc:511] Master 127.31.43.254:34489 requested a full tablet report, sending...
I20260812 03:57:40.734766 3022 ts_manager.cc:194] Registered new tserver with Master: cb9ef779b4b8470583105f42a901cf3c (127.31.43.195:38489)
I20260812 03:57:40.735328 3022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:54265
I20260812 03:57:40.735519 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[2968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507060, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[2968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507060, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:40.763265 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:40.754758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57756 (local address 127.31.43.254:34489)
0812 03:57:40.754829 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:40.754831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:40.755058 (+ 227us) server_negotiation.cc:407] Connection header received
0812 03:57:40.755095 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:40.755096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:40.755108 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:40.755157 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:40.755891 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.756497 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.757289 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:40.757485 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:40.757545 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:40.760343 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:40.760345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:40.760345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:40.760358 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:40.761877 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:40.762306 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:40.762307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:40.762307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:40.762366 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:40.762602 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:40.762603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:40.762603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:40.762816 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:40.762897 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:40.763122 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 03:57:40.763194 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 03:57:40.763552 3022 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57756
I20260812 03:57:40.764246 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 3091
I20260812 03:57:40.772701 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 3248
I20260812 03:57:40.780411 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 3407
I20260812 03:57:40.788004 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 2983
2026-08-12T03:57:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2083 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[3562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[3562](info): set up 2 sockets
Aug 12 03:57:40 dist-test-slave-8tdl krb5kdc[3562](info): commencing operation
krb5kdc: starting...
W20260812 03:57:41.996495 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260812 03:57:42.217002 31919 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.220s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260812 03:57:42.447134 31919 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.230s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260812 03:57:42.678194 31919 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.231s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/test-user.keytab.
W20260812 03:57:43.353052 31919 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.675s user 0.000s sys 0.002s
Aug 12 03:57:43 dist-test-slave-8tdl krb5kdc[3562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:57:43.410636 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:46831
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:46831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:43.500113 3579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:43.500375 3579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:43.500401 3579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:43.501684 3579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:43.501744 3579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:43.501760 3579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:43.501772 3579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:43.501784 3579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:43.503538 3579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:46831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:46831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:43.503851 3579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:43.504127 3579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:43.509619 3584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:43.509593 3588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:43.509629 3585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:43.510070 3579 server_base.cc:1034] running on GCE node
I20260812 03:57:43.510354 3579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:43.510756 3579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:43.511925 3579 hybrid_clock.cc:648] HybridClock initialized: now 1786507063511902 us; error 48 us; skew 500 ppm
Aug 12 03:57:43 dist-test-slave-8tdl krb5kdc[3562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507063, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:43.515182 3579 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:57:43.515864 3579 webserver.cc:460] Webserver started at http://127.31.43.254:41053/ using document root <none> and password file <none>
I20260812 03:57:43.516180 3579 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:43.516258 3579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:43.516425 3579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:43.517880 3579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "b582bc95b4414904abea776aa3aec12a"
format_stamp: "Formatted at 2026-08-12 03:57:43 on dist-test-slave-8tdl"
server_key: "6cb8ae98a4da808c215c0644b4886f8c"
server_key_iv: "8001b7475e695a98ceb4e4689c4d4fe7"
server_key_version: "encryptionkey@0"
I20260812 03:57:43.518386 3579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "b582bc95b4414904abea776aa3aec12a"
format_stamp: "Formatted at 2026-08-12 03:57:43 on dist-test-slave-8tdl"
server_key: "6cb8ae98a4da808c215c0644b4886f8c"
server_key_iv: "8001b7475e695a98ceb4e4689c4d4fe7"
server_key_version: "encryptionkey@0"
I20260812 03:57:43.520304 3579 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 03:57:43.521380 3595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:43.521765 3579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:43.521886 3579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "b582bc95b4414904abea776aa3aec12a"
format_stamp: "Formatted at 2026-08-12 03:57:43 on dist-test-slave-8tdl"
server_key: "6cb8ae98a4da808c215c0644b4886f8c"
server_key_iv: "8001b7475e695a98ceb4e4689c4d4fe7"
server_key_version: "encryptionkey@0"
I20260812 03:57:43.521993 3579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:43.538008 3579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:43.540695 3579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:43.540891 3579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:43.545702 3579 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:46831
I20260812 03:57:43.545763 3654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:46831 every 8 connection(s)
I20260812 03:57:43.546366 3579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:43.547151 3655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:43.547487 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 3579
I20260812 03:57:43.547610 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:43.547801 31919 external_mini_cluster.cc:1428] Setting key 469284b28ef0aaa60b762c6e9ea245a6
I20260812 03:57:43.550520 3655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a: Bootstrap starting.
I20260812 03:57:43.551235 3655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:43.551590 3655 log.cc:824] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:43.552464 3655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a: No bootstrap required, opened a new log
Aug 12 03:57:43 dist-test-slave-8tdl krb5kdc[3562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:43.555342 3655 raft_consensus.cc:348] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b582bc95b4414904abea776aa3aec12a" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 46831 } }
I20260812 03:57:43.555497 3655 raft_consensus.cc:374] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:43.555521 3655 raft_consensus.cc:729] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b582bc95b4414904abea776aa3aec12a, State: Initialized, Role: FOLLOWER
I20260812 03:57:43.555680 3655 consensus_queue.cc:260] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b582bc95b4414904abea776aa3aec12a" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 46831 } }
I20260812 03:57:43.555776 3655 raft_consensus.cc:388] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:43.555823 3655 raft_consensus.cc:482] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:43.555877 3655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:43.556468 3655 raft_consensus.cc:504] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b582bc95b4414904abea776aa3aec12a" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 46831 } }
I20260812 03:57:43.556604 3655 leader_election.cc:302] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b582bc95b4414904abea776aa3aec12a; no voters:
I20260812 03:57:43.556818 3655 leader_election.cc:288] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:43.556915 3661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:43.557078 3661 raft_consensus.cc:686] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [term 1 LEADER]: Becoming Leader. State: Replica: b582bc95b4414904abea776aa3aec12a, State: Running, Role: LEADER
I20260812 03:57:43.557294 3661 consensus_queue.cc:237] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b582bc95b4414904abea776aa3aec12a" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 46831 } }
I20260812 03:57:43.557349 3655 sys_catalog.cc:564] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:43.557785 3662 sys_catalog.cc:455] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b582bc95b4414904abea776aa3aec12a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b582bc95b4414904abea776aa3aec12a" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 46831 } } }
I20260812 03:57:43.557871 3662 sys_catalog.cc:458] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:43.558125 3665 sys_catalog.cc:455] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b582bc95b4414904abea776aa3aec12a. Latest consensus state: current_term: 1 leader_uuid: "b582bc95b4414904abea776aa3aec12a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b582bc95b4414904abea776aa3aec12a" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 46831 } } }
I20260812 03:57:43.558269 3665 sys_catalog.cc:458] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:43.558454 3668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:43.559072 3668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:43.560257 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:43.548391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44634 (local address 127.31.43.254:46831)
0812 03:57:43.548586 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:43.548595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:43.548623 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:57:43.549049 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:43.549056 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:43.549123 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:43.549256 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:43.550138 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:43.551230 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:43.552251 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:43.552754 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:43.552859 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:43.555686 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:43.555691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:43.555691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:43.555712 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:43.558050 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:43.558517 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:43.558520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:43.558520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:43.558636 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:43.558928 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:43.558929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:43.558929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:43.559252 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:43.559888 (+ 636us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:43.559966 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 03:57:43.560075 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260812 03:57:43.560873 3668 catalog_manager.cc:1347] Generated new cluster ID: 1e619f1be2f04316a66b36d4e55089e0
I20260812 03:57:43.560933 3668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:43.566816 3668 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:43.567535 3668 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:43.582135 3668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b582bc95b4414904abea776aa3aec12a: Generated new TSK 0
I20260812 03:57:43.582393 3668 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 03:57:43.614842 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:46831
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:43.705508 3692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:43.705829 3692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:43.705940 3692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:43.707592 3692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:43.707669 3692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:43.707717 3692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:43.710160 3692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:46831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:43.710599 3692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:43.710937 3692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:43.718129 3697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:43.718195 3700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:43.718145 3698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:43.718958 3692 server_base.cc:1034] running on GCE node
I20260812 03:57:43.719203 3692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:43.719594 3692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:43.720773 3692 hybrid_clock.cc:648] HybridClock initialized: now 1786507063720730 us; error 56 us; skew 500 ppm
Aug 12 03:57:43 dist-test-slave-8tdl krb5kdc[3562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507063, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:43.724654 3692 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 03:57:43.725340 3692 webserver.cc:460] Webserver started at http://127.31.43.193:43221/ using document root <none> and password file <none>
I20260812 03:57:43.725695 3692 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:43.725775 3692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:43.725952 3692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:43.727715 3692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "cec79149b6b54b608c3db7ef9da5cf3b"
format_stamp: "Formatted at 2026-08-12 03:57:43 on dist-test-slave-8tdl"
server_key: "7e7e1b34128ae47c578f61ea7cc4ab70"
server_key_iv: "4ffe5163cb7c801e3156b65605b9a3a0"
server_key_version: "encryptionkey@0"
I20260812 03:57:43.728184 3692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "cec79149b6b54b608c3db7ef9da5cf3b"
format_stamp: "Formatted at 2026-08-12 03:57:43 on dist-test-slave-8tdl"
server_key: "7e7e1b34128ae47c578f61ea7cc4ab70"
server_key_iv: "4ffe5163cb7c801e3156b65605b9a3a0"
server_key_version: "encryptionkey@0"
I20260812 03:57:43.729977 3692 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:57:43.731173 3707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:43.731385 3692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:57:43.731504 3692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "cec79149b6b54b608c3db7ef9da5cf3b"
format_stamp: "Formatted at 2026-08-12 03:57:43 on dist-test-slave-8tdl"
server_key: "7e7e1b34128ae47c578f61ea7cc4ab70"
server_key_iv: "4ffe5163cb7c801e3156b65605b9a3a0"
server_key_version: "encryptionkey@0"
I20260812 03:57:43.731616 3692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:43.744292 3692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:43.746934 3692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:43.747113 3692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:43.747381 3692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:43.747749 3692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:43.747807 3692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:43.747853 3692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:43.747882 3692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:43.755787 3692 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:33041
I20260812 03:57:43.755854 3832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:33041 every 8 connection(s)
I20260812 03:57:43.756287 3692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:57:43.761329 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 3692
I20260812 03:57:43.761468 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:57:43.761642 31919 external_mini_cluster.cc:1428] Setting key 5454311e38a0ce567da54bc056ee815a
Aug 12 03:57:43 dist-test-slave-8tdl krb5kdc[3562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507063, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:43.769083 3836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:43.757232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:46831 (local address 127.31.43.193:40733)
0812 03:57:43.757418 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:43.757475 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:43.758995 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:43.759326 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:43.759332 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:43.759771 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:43.760129 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:43.760136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:43.760846 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:43.760848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:43.762280 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:43.762310 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:43.762755 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:43.762757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:43.762905 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:43.763472 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:43.763489 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:43.765555 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:43.767342 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:43.767353 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:43.767356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:43.767747 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:43.767977 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:43.767979 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:43.767979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:43.768144 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:43.768449 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:43.768455 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:43.768630 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 03:57:43.768727 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:57:43.768874 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260812 03:57:43.769151 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:43.757266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:40733 (local address 127.31.43.254:46831)
0812 03:57:43.757328 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:43.757331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:43.757589 (+ 258us) server_negotiation.cc:407] Connection header received
0812 03:57:43.759128 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:43.759130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:43.759142 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:43.759228 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:43.760238 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:43.760764 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:43.762394 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:43.762603 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:43.762661 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:43.765690 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:43.765692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:43.765692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:43.765700 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:43.767247 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:43.767830 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:43.767831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:43.767831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:43.767883 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:43.768242 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:43.768243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:43.768244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:43.768373 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:43.768445 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:43.768796 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 03:57:43.768919 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:57:43.769707 3834 heartbeater.cc:347] Connected to a master server at 127.31.43.254:46831
I20260812 03:57:43.769835 3834 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:43.770061 3834 heartbeater.cc:511] Master 127.31.43.254:46831 requested a full tablet report, sending...
I20260812 03:57:43.770780 3613 ts_manager.cc:194] Registered new tserver with Master: cec79149b6b54b608c3db7ef9da5cf3b (127.31.43.193:33041)
I20260812 03:57:43.771790 3613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:40733
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 03:57:43.795871 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:46831
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:43.883535 3839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:43.883803 3839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:43.883845 3839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:43.885062 3839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:43.885145 3839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:43.885192 3839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:43.886912 3839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:46831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:43.887210 3839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:43.887456 3839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:43.893124 3846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:43.893134 3845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:43.893414 3848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:43.893703 3839 server_base.cc:1034] running on GCE node
I20260812 03:57:43.893929 3839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:43.894212 3839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:43.895386 3839 hybrid_clock.cc:648] HybridClock initialized: now 1786507063895348 us; error 57 us; skew 500 ppm
Aug 12 03:57:43 dist-test-slave-8tdl krb5kdc[3562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507063, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:43.898411 3839 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 03:57:43.899016 3839 webserver.cc:460] Webserver started at http://127.31.43.194:36039/ using document root <none> and password file <none>
I20260812 03:57:43.899286 3839 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:43.899362 3839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:43.899523 3839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:43.900665 3839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "7de4adba1c594d3ebdf74c1314b11d40"
format_stamp: "Formatted at 2026-08-12 03:57:43 on dist-test-slave-8tdl"
server_key: "e9e7d9e47d09c43fab9800112147b67e"
server_key_iv: "f50acab97068e7af0daa994be7b9c7bc"
server_key_version: "encryptionkey@0"
I20260812 03:57:43.901053 3839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "7de4adba1c594d3ebdf74c1314b11d40"
format_stamp: "Formatted at 2026-08-12 03:57:43 on dist-test-slave-8tdl"
server_key: "e9e7d9e47d09c43fab9800112147b67e"
server_key_iv: "f50acab97068e7af0daa994be7b9c7bc"
server_key_version: "encryptionkey@0"
I20260812 03:57:43.902622 3839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:43.903481 3855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:43.903689 3839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:43.903812 3839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "7de4adba1c594d3ebdf74c1314b11d40"
format_stamp: "Formatted at 2026-08-12 03:57:43 on dist-test-slave-8tdl"
server_key: "e9e7d9e47d09c43fab9800112147b67e"
server_key_iv: "f50acab97068e7af0daa994be7b9c7bc"
server_key_version: "encryptionkey@0"
I20260812 03:57:43.903895 3839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:43.916033 3839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:43.918308 3839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:43.918597 3839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:43.918962 3839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:43.919317 3839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:43.919375 3839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:43.919422 3839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:43.919456 3839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:43.926563 3839 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:43323
I20260812 03:57:43.926607 3987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:43323 every 8 connection(s)
I20260812 03:57:43.927052 3839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:57:43.932616 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 3839
I20260812 03:57:43.932786 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:57:43.933413 31919 external_mini_cluster.cc:1428] Setting key c3cdf3ce5723ee1581b22a3b0b6d9c54
Aug 12 03:57:43 dist-test-slave-8tdl krb5kdc[3562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507063, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:43.939018 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:43.927947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:50219 (local address 127.31.43.254:46831)
0812 03:57:43.928039 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:43.928041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:43.928285 (+ 244us) server_negotiation.cc:407] Connection header received
0812 03:57:43.929268 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:43.929269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:43.929284 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:43.929356 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:43.930044 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:43.930580 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:43.931970 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:43.932181 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:43.932247 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:43.935427 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:43.935429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:43.935429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:43.935436 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:43.937279 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:43.937789 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:43.937790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:43.937791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:43.937856 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:43.938164 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:43.938165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:43.938165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:43.938327 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:43.938398 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:43.938756 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 03:57:43.938857 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:57:43.939178 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:43.927919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:46831 (local address 127.31.43.194:50219)
0812 03:57:43.928117 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:43.928164 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:43.929144 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:43.929438 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:43.929442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:43.929735 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:43.929943 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:43.929947 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:43.930718 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:43.930720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:43.931847 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:43.931851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:43.932318 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:43.932319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:43.932538 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:43.933250 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:43.933262 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:43.935309 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:43.937388 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:43.937397 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:43.937399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:43.937701 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:43.937928 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:43.937929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:43.937929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:43.938069 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:43.938415 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:43.938422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:43.938571 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 03:57:43.938669 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:57:43.938901 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":118,"threads_started":1}
I20260812 03:57:43.939750 3988 heartbeater.cc:347] Connected to a master server at 127.31.43.254:46831
I20260812 03:57:43.939878 3988 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:43.940107 3988 heartbeater.cc:511] Master 127.31.43.254:46831 requested a full tablet report, sending...
I20260812 03:57:43.940686 3613 ts_manager.cc:194] Registered new tserver with Master: 7de4adba1c594d3ebdf74c1314b11d40 (127.31.43.194:43323)
I20260812 03:57:43.941270 3613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:50219
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 03:57:43.967352 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:46831
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:44.054176 3995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:44.054447 3995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:44.054489 3995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:44.055735 3995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:44.055814 3995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:44.055866 3995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:44.057497 3995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:46831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:44.057886 3995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:44.058161 3995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:44.063720 4001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:44.063720 4002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:44.063773 4004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:44.064100 3995 server_base.cc:1034] running on GCE node
I20260812 03:57:44.064378 3995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:44.064658 3995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:44.065830 3995 hybrid_clock.cc:648] HybridClock initialized: now 1786507064065798 us; error 51 us; skew 500 ppm
Aug 12 03:57:44 dist-test-slave-8tdl krb5kdc[3562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507064, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:44.068764 3995 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 03:57:44.069341 3995 webserver.cc:460] Webserver started at http://127.31.43.195:44085/ using document root <none> and password file <none>
I20260812 03:57:44.069607 3995 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:44.069657 3995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:44.069796 3995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:44.070955 3995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "3df15743dd4a42499be8dcb998f2446c"
format_stamp: "Formatted at 2026-08-12 03:57:44 on dist-test-slave-8tdl"
server_key: "7a6ac3d5ec57a43b3e61047ec9862c32"
server_key_iv: "030f556ac5fb0c2b87ebd91f91db397e"
server_key_version: "encryptionkey@0"
I20260812 03:57:44.071328 3995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "3df15743dd4a42499be8dcb998f2446c"
format_stamp: "Formatted at 2026-08-12 03:57:44 on dist-test-slave-8tdl"
server_key: "7a6ac3d5ec57a43b3e61047ec9862c32"
server_key_iv: "030f556ac5fb0c2b87ebd91f91db397e"
server_key_version: "encryptionkey@0"
I20260812 03:57:44.072801 3995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:57:44.073671 4015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:44.073876 3995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:44.073975 3995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "3df15743dd4a42499be8dcb998f2446c"
format_stamp: "Formatted at 2026-08-12 03:57:44 on dist-test-slave-8tdl"
server_key: "7a6ac3d5ec57a43b3e61047ec9862c32"
server_key_iv: "030f556ac5fb0c2b87ebd91f91db397e"
server_key_version: "encryptionkey@0"
I20260812 03:57:44.074052 3995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:44.090961 3995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:44.093160 3995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:44.093315 3995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:44.093572 3995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:44.093909 3995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:44.093964 3995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:44.094007 3995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:44.094034 3995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:44.100476 3995 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:32823
I20260812 03:57:44.100521 4142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:32823 every 8 connection(s)
I20260812 03:57:44.100875 3995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:57:44.104280 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 3995
I20260812 03:57:44.104405 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:57:44.104592 31919 external_mini_cluster.cc:1428] Setting key 5040e9ffc67d8e11144b2e54e3ac0618
Aug 12 03:57:44 dist-test-slave-8tdl krb5kdc[3562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507064, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:44.112462 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:44.101712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:49121 (local address 127.31.43.254:46831)
0812 03:57:44.101783 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:44.101786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:44.102032 (+ 246us) server_negotiation.cc:407] Connection header received
0812 03:57:44.103246 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:44.103247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:44.103260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:44.103329 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:44.104115 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:44.104742 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:44.106223 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:44.106409 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:44.106466 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:44.108999 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:44.109001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:44.109001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:44.109008 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:44.110476 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:44.111204 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:44.111205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:44.111206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:44.111270 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:44.111596 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:44.111597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:44.111598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:44.111828 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:44.111892 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:44.112232 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 03:57:44.112338 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:57:44.112636 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:44.101679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:46831 (local address 127.31.43.195:49121)
0812 03:57:44.101886 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:44.101934 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:44.103137 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:44.103406 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:44.103409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:44.103739 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:44.103964 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:44.103969 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:44.104881 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:44.104882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:44.106135 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:44.106139 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:44.106503 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:44.106504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:44.106622 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:44.107296 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:44.107306 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:44.108894 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:44.110571 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:44.110579 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:44.110581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:44.111127 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:44.111351 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:44.111352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:44.111353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:44.111514 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:44.111902 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:44.111910 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:44.112084 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 03:57:44.112202 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 03:57:44.112380 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":124,"threads_started":1}
I20260812 03:57:44.113319 4143 heartbeater.cc:347] Connected to a master server at 127.31.43.254:46831
I20260812 03:57:44.113445 4143 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:44.113672 4143 heartbeater.cc:511] Master 127.31.43.254:46831 requested a full tablet report, sending...
I20260812 03:57:44.114076 3613 ts_manager.cc:194] Registered new tserver with Master: 3df15743dd4a42499be8dcb998f2446c (127.31.43.195:32823)
I20260812 03:57:44.114468 3613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:49121
I20260812 03:57:44.116461 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:57:44 dist-test-slave-8tdl krb5kdc[3562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:57:44 dist-test-slave-8tdl krb5kdc[3562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.193@KRBTEST.COM
I20260812 03:57:44.143375 4150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:44.134338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40210 (local address 127.31.43.193:33041)
0812 03:57:44.134571 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:44.134576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:44.134588 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:57:44.134618 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:44.134620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:44.134634 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:44.134787 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:44.135531 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:44.136377 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:44.137252 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:44.137592 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:44.137649 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:44.140409 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:44.140421 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:44.140422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:44.140440 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:44.142138 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:44.142497 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:44.142500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:44.142501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:44.142577 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:44.142836 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:44.142836 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:44.142837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:44.143014 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:44.143153 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:44.143213 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 03:57:44.143286 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":142,"threads_started":1}
Aug 12 03:57:44 dist-test-slave-8tdl krb5kdc[3562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:44.152597 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:44.145083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44648 (local address 127.31.43.254:46831)
0812 03:57:44.145166 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:44.145169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:44.145180 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:57:44.145225 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:44.145226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:44.145236 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:44.145289 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:44.145792 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:44.146351 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:44.147143 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:44.147386 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:44.147474 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:44.149792 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:44.149793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:44.149794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:44.149802 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:44.151421 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:44.151741 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:44.151742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:44.151742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:44.151798 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:44.152002 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:44.152003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:44.152003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:44.152157 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:44.152240 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:44.152337 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 03:57:44.152404 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 03:57:44.152912 3613 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44648
I20260812 03:57:44.153414 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 3692
I20260812 03:57:44.161473 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 3839
I20260812 03:57:44.168349 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 3995
I20260812 03:57:44.176997 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 3579
2026-08-12T03:57:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3388 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:44 dist-test-slave-8tdl krb5kdc[4157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:44 dist-test-slave-8tdl krb5kdc[4157](info): set up 2 sockets
Aug 12 03:57:44 dist-test-slave-8tdl krb5kdc[4157](info): commencing operation
krb5kdc: starting...
W20260812 03:57:45.330608 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:57:45 dist-test-slave-8tdl krb5kdc[4157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:57:45.475716 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:43931
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:38719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:43931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:45.568821 4174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:45.569157 4174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:45.569203 4174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:45.570550 4174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:45.570637 4174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:45.570660 4174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:45.570681 4174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:45.570768 4174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:45.572532 4174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:38719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:43931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:43931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:45.572873 4174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:45.573313 4174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:45.579784 4182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:45.579784 4184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:45.579784 4181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:45.580284 4174 server_base.cc:1034] running on GCE node
I20260812 03:57:45.580668 4174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:45.581033 4174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:45.582230 4174 hybrid_clock.cc:648] HybridClock initialized: now 1786507065582185 us; error 88 us; skew 500 ppm
Aug 12 03:57:45 dist-test-slave-8tdl krb5kdc[4157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507065, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:45.585428 4174 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:57:45.585897 4174 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:45.585966 4174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:45.586090 4174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:45.587422 4174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "71c038842dbd4f4681abe94b6d54d4ff"
format_stamp: "Formatted at 2026-08-12 03:57:45 on dist-test-slave-8tdl"
server_key: "9c0ae44b9291c562b052e0ee34431377"
server_key_iv: "77a77988819150bde6f943890bcbe460"
server_key_version: "encryptionkey@0"
I20260812 03:57:45.587796 4174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "71c038842dbd4f4681abe94b6d54d4ff"
format_stamp: "Formatted at 2026-08-12 03:57:45 on dist-test-slave-8tdl"
server_key: "9c0ae44b9291c562b052e0ee34431377"
server_key_iv: "77a77988819150bde6f943890bcbe460"
server_key_version: "encryptionkey@0"
I20260812 03:57:45.589391 4174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:57:45.590534 4191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:45.590821 4174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:45.590938 4174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "71c038842dbd4f4681abe94b6d54d4ff"
format_stamp: "Formatted at 2026-08-12 03:57:45 on dist-test-slave-8tdl"
server_key: "9c0ae44b9291c562b052e0ee34431377"
server_key_iv: "77a77988819150bde6f943890bcbe460"
server_key_version: "encryptionkey@0"
I20260812 03:57:45.591008 4174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:45.601770 4174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:45.604245 4174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:45.604398 4174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:45.608865 4174 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:43931
I20260812 03:57:45.608913 4247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:43931 every 8 connection(s)
I20260812 03:57:45.609253 4174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:45.610472 4248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:45.612500 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 4174
I20260812 03:57:45.612620 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:45.612856 31919 external_mini_cluster.cc:1428] Setting key b620ce61b8bbef489a78cac41e69395d
I20260812 03:57:45.613453 4248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff: Bootstrap starting.
I20260812 03:57:45.614254 4248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:45.614609 4248 log.cc:824] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:45.615463 4248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff: No bootstrap required, opened a new log
I20260812 03:57:45.617367 4248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71c038842dbd4f4681abe94b6d54d4ff" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43931 } }
I20260812 03:57:45.617566 4248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:45.617638 4248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71c038842dbd4f4681abe94b6d54d4ff, State: Initialized, Role: FOLLOWER
I20260812 03:57:45.617805 4248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71c038842dbd4f4681abe94b6d54d4ff" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43931 } }
I20260812 03:57:45.617895 4248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:45.617926 4248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:45.617965 4248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:45.618898 4248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71c038842dbd4f4681abe94b6d54d4ff" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43931 } }
I20260812 03:57:45.619026 4248 leader_election.cc:302] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71c038842dbd4f4681abe94b6d54d4ff; no voters:
I20260812 03:57:45.619244 4248 leader_election.cc:288] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:45.619292 4253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:45.619412 4253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [term 1 LEADER]: Becoming Leader. State: Replica: 71c038842dbd4f4681abe94b6d54d4ff, State: Running, Role: LEADER
I20260812 03:57:45.619503 4253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71c038842dbd4f4681abe94b6d54d4ff" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43931 } }
I20260812 03:57:45.619567 4248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:45.619849 4253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71c038842dbd4f4681abe94b6d54d4ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71c038842dbd4f4681abe94b6d54d4ff" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43931 } } }
I20260812 03:57:45.619925 4253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:45.620079 4253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71c038842dbd4f4681abe94b6d54d4ff. Latest consensus state: current_term: 1 leader_uuid: "71c038842dbd4f4681abe94b6d54d4ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71c038842dbd4f4681abe94b6d54d4ff" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43931 } } }
I20260812 03:57:45.620152 4253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:45.620464 4257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:45.621110 4257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:57:45 dist-test-slave-8tdl krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:45.623358 4257 catalog_manager.cc:1347] Generated new cluster ID: 335a55a4fc904ee18a75295dc7a818e3
I20260812 03:57:45.623425 4257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:45.625765 4251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:45.613612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55960 (local address 127.31.43.254:43931)
0812 03:57:45.613833 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:45.613842 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:45.613872 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:57:45.614633 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:45.614642 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:45.614762 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:45.614965 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:45.615736 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:45.616743 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:45.617709 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:45.618375 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:45.618508 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:45.621958 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:45.621963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:45.621963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:45.621977 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:45.624031 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:45.624453 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:45.624456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:45.624457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:45.624558 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:45.624789 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:45.624790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:45.624790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:45.625052 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:45.625283 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:45.625357 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:57:45.625506 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":121,"threads_started":1}
I20260812 03:57:45.628345 4257 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:45.628963 4257 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:45.637106 4257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff: Generated new TSK 0
I20260812 03:57:45.637377 4257 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 03:57:45.683283 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43931
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:38719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:45.804360 4278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:45.804646 4278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:45.804680 4278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:45.805997 4278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:45.806056 4278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:45.806090 4278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:45.807827 4278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:38719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:45.808100 4278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:45.808380 4278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:45.813951 4289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:45.813949 4285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:45.814051 4284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:45.814177 4278 server_base.cc:1034] running on GCE node
I20260812 03:57:45.814536 4278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:45.814848 4278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:45.816038 4278 hybrid_clock.cc:648] HybridClock initialized: now 1786507065816026 us; error 36 us; skew 500 ppm
Aug 12 03:57:45 dist-test-slave-8tdl krb5kdc[4157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507065, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:45.818852 4278 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 03:57:45.819195 4278 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:45.819242 4278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:45.819342 4278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:45.820562 4278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "56e8135d0a1c46afb5134962759712e5"
format_stamp: "Formatted at 2026-08-12 03:57:45 on dist-test-slave-8tdl"
server_key: "7ec179fae0c56f4134f81e35dfa8a71a"
server_key_iv: "74fc2a1393fc80a336d37ffb8cb34dfd"
server_key_version: "encryptionkey@0"
I20260812 03:57:45.820976 4278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "56e8135d0a1c46afb5134962759712e5"
format_stamp: "Formatted at 2026-08-12 03:57:45 on dist-test-slave-8tdl"
server_key: "7ec179fae0c56f4134f81e35dfa8a71a"
server_key_iv: "74fc2a1393fc80a336d37ffb8cb34dfd"
server_key_version: "encryptionkey@0"
I20260812 03:57:45.822412 4278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:45.823377 4295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:45.823628 4278 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:45.823721 4278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "56e8135d0a1c46afb5134962759712e5"
format_stamp: "Formatted at 2026-08-12 03:57:45 on dist-test-slave-8tdl"
server_key: "7ec179fae0c56f4134f81e35dfa8a71a"
server_key_iv: "74fc2a1393fc80a336d37ffb8cb34dfd"
server_key_version: "encryptionkey@0"
I20260812 03:57:45.823786 4278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:45.856001 4278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:45.858596 4278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:45.859020 4278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:45.859333 4278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:45.859807 4278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:45.859854 4278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:45.859882 4278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:45.859896 4278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:45.867506 4278 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:45133
I20260812 03:57:45.867547 4421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:45133 every 8 connection(s)
I20260812 03:57:45.867924 4278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:57:45.871822 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 4278
I20260812 03:57:45.871937 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:57:45.872159 31919 external_mini_cluster.cc:1428] Setting key 54eb53d0caef456b1ed2341ff5828d30
Aug 12 03:57:45 dist-test-slave-8tdl krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507065, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:45.880540 4251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:45.868847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:60071 (local address 127.31.43.254:43931)
0812 03:57:45.868916 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:45.868919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:45.869174 (+ 255us) server_negotiation.cc:407] Connection header received
0812 03:57:45.869745 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:45.869746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:45.869763 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:45.869817 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:45.871179 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:45.871786 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:45.873285 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:45.873519 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:45.873598 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:45.876553 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:45.876555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:45.876555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:45.876566 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:45.878610 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:45.879301 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:45.879303 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:45.879303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:45.879385 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:45.879684 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:45.879685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:45.879686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:45.879863 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:45.879950 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:45.880292 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 03:57:45.880422 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:57:45.880661 4424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:45.868824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:43931 (local address 127.31.43.193:60071)
0812 03:57:45.869018 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:45.869066 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:45.869643 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:45.869890 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:45.869893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:45.870199 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:45.871050 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:45.871058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:45.871930 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:45.871932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:45.873185 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:45.873189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:45.873612 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:45.873613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:45.873751 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:45.874510 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:45.874519 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:45.876414 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:45.878788 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:45.878802 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:45.878805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:45.879199 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:45.879475 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:45.879475 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:45.879476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:45.879607 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:45.879965 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:45.879973 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:45.880134 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 03:57:45.880242 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 03:57:45.880416 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
I20260812 03:57:45.881316 4422 heartbeater.cc:347] Connected to a master server at 127.31.43.254:43931
I20260812 03:57:45.881451 4422 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:45.881670 4422 heartbeater.cc:511] Master 127.31.43.254:43931 requested a full tablet report, sending...
I20260812 03:57:45.882510 4211 ts_manager.cc:194] Registered new tserver with Master: 56e8135d0a1c46afb5134962759712e5 (127.31.43.193:45133)
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
I20260812 03:57:45.883490 4211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:60071
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 03:57:45.907721 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43931
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:38719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:45.997391 4428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:45.997645 4428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:45.997673 4428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:45.999030 4428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:45.999084 4428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:45.999114 4428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:46.000820 4428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:38719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:46.001116 4428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:46.001449 4428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:46.007495 4435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:46.007553 4436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:46.007530 4438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:46.007720 4428 server_base.cc:1034] running on GCE node
I20260812 03:57:46.008008 4428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:46.008313 4428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:46.009481 4428 hybrid_clock.cc:648] HybridClock initialized: now 1786507066009444 us; error 53 us; skew 500 ppm
Aug 12 03:57:46 dist-test-slave-8tdl krb5kdc[4157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507066, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:46.012605 4428 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 03:57:46.013073 4428 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:46.013155 4428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:46.013396 4428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:46.014595 4428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "34ab76e815cd46faabb73959920bc806"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "f036c88111a5ae8eca943cecb6e00190"
server_key_iv: "5bae27fc5eea0fb883a6a33fca9a72cc"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.015085 4428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "34ab76e815cd46faabb73959920bc806"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "f036c88111a5ae8eca943cecb6e00190"
server_key_iv: "5bae27fc5eea0fb883a6a33fca9a72cc"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.016583 4428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:57:46.017596 4446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.017838 4428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:46.017961 4428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "34ab76e815cd46faabb73959920bc806"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "f036c88111a5ae8eca943cecb6e00190"
server_key_iv: "5bae27fc5eea0fb883a6a33fca9a72cc"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.018049 4428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:46.031973 4428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:46.034741 4428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:46.034950 4428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:46.035231 4428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:46.035598 4428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:46.035668 4428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.035706 4428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:46.035751 4428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.042348 4428 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:40515
I20260812 03:57:46.042407 4571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:40515 every 8 connection(s)
I20260812 03:57:46.042788 4428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:57:46.044487 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 4428
I20260812 03:57:46.044636 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:57:46.044826 31919 external_mini_cluster.cc:1428] Setting key da1ce2ab3b8f84a4e0be16c69cca2bba
Aug 12 03:57:46 dist-test-slave-8tdl krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507066, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
I20260812 03:57:46.054119 4251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.043772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:58479 (local address 127.31.43.254:43931)
0812 03:57:46.043852 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.043854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.044078 (+ 224us) server_negotiation.cc:407] Connection header received
0812 03:57:46.044648 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.044649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.044664 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.044712 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.045968 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.046479 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.047896 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.048100 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.048169 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.050926 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.050928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.050929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:46.050940 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.052514 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:46.053031 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:46.053032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:46.053032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:46.053099 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:46.053383 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:46.053384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:46.053385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:46.053528 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.053605 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.053930 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.054020 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:57:46.054349 4574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.043739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:43931 (local address 127.31.43.194:58479)
0812 03:57:46.043936 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.043981 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.044526 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.044808 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.044812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.045255 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:46.045868 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.045874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.046587 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.046588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.047777 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.047781 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.048195 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.048196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.048347 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.048985 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:46.048995 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:46.050810 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:46.052657 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:46.052667 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:46.052669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:46.052933 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:46.053177 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:46.053177 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:46.053178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:46.053304 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:46.053608 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:46.053617 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:46.053782 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.053892 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.054077 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":119,"threads_started":1}
I20260812 03:57:46.054894 4572 heartbeater.cc:347] Connected to a master server at 127.31.43.254:43931
I20260812 03:57:46.055022 4572 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:46.055200 4572 heartbeater.cc:511] Master 127.31.43.254:43931 requested a full tablet report, sending...
I20260812 03:57:46.055619 4211 ts_manager.cc:194] Registered new tserver with Master: 34ab76e815cd46faabb73959920bc806 (127.31.43.194:40515)
I20260812 03:57:46.056027 4211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:58479
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 03:57:46.078959 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43931
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:38719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:46.169379 4577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:46.169660 4577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:46.169701 4577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:46.171080 4577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:46.171238 4577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:46.171350 4577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:46.173187 4577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:38719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:46.173517 4577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:46.173779 4577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:46.179252 4587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:46.179374 4577 server_base.cc:1034] running on GCE node
W20260812 03:57:46.179256 4583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:46.179252 4584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:46.179814 4577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:46.180141 4577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:46.181319 4577 hybrid_clock.cc:648] HybridClock initialized: now 1786507066181304 us; error 34 us; skew 500 ppm
Aug 12 03:57:46 dist-test-slave-8tdl krb5kdc[4157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507066, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:46.184219 4577 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 03:57:46.184644 4577 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:46.184701 4577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:46.184803 4577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:46.185948 4577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "6edf733b72f344edb2f172eb020ff2ea"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "4db77c239c693bf564ae154601cb4959"
server_key_iv: "29c061f02ad0271ad2faffca76700492"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.186295 4577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "6edf733b72f344edb2f172eb020ff2ea"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "4db77c239c693bf564ae154601cb4959"
server_key_iv: "29c061f02ad0271ad2faffca76700492"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.187839 4577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:57:46.188762 4593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.189090 4577 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:46.189230 4577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "6edf733b72f344edb2f172eb020ff2ea"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "4db77c239c693bf564ae154601cb4959"
server_key_iv: "29c061f02ad0271ad2faffca76700492"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.189320 4577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:46.204525 4577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:46.207074 4577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:46.207261 4577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:46.207538 4577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:46.207891 4577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:46.207952 4577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.207993 4577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:46.208024 4577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.214475 4577 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:37581
I20260812 03:57:46.214520 4725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:37581 every 8 connection(s)
I20260812 03:57:46.214903 4577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:57:46.215709 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 4577
I20260812 03:57:46.215823 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:57:46.216023 31919 external_mini_cluster.cc:1428] Setting key 679d5609b64311df4e843f6c2be16373
Aug 12 03:57:46 dist-test-slave-8tdl krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507066, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:46.226056 4251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.216218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:50955 (local address 127.31.43.254:43931)
0812 03:57:46.216289 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.216291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.216302 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:57:46.216968 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.216970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.216985 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.217041 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.218116 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.218657 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.219750 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.219988 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.220113 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.222731 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.222733 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.222733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:46.222743 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.224312 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:46.224877 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:46.224878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:46.224879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:46.224955 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:46.225284 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:46.225285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:46.225286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:46.225426 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.225514 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.225847 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.225947 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:57:46.226296 4728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.215830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:43931 (local address 127.31.43.195:50955)
0812 03:57:46.216024 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.216102 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.216820 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.217136 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.217146 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.217431 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:46.218001 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.218006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.218794 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.218795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.219632 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.219635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.220149 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.220150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.220285 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.220879 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:46.220888 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:46.222582 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:46.224456 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:46.224473 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:46.224476 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:46.224780 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:46.225052 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:46.225053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:46.225054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:46.225195 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:46.225504 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:46.225512 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:46.225675 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.225812 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.226011 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
I20260812 03:57:46.226923 4726 heartbeater.cc:347] Connected to a master server at 127.31.43.254:43931
I20260812 03:57:46.227029 4726 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:46.227269 4726 heartbeater.cc:511] Master 127.31.43.254:43931 requested a full tablet report, sending...
I20260812 03:57:46.227846 4211 ts_manager.cc:194] Registered new tserver with Master: 6edf733b72f344edb2f172eb020ff2ea (127.31.43.195:37581)
I20260812 03:57:46.228053 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:57:46.228348 4211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:50955
I20260812 03:57:46.235991 4251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.229616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55966 (local address 127.31.43.254:43931)
0812 03:57:46.229672 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.229675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.229711 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:57:46.229766 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.229767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.229776 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.229828 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.230417 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.231034 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.231808 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.232006 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.232071 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.233389 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.233390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.233390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:46.233397 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.234826 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:46.235208 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:46.235209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:46.235209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:46.235267 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:46.235444 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:46.235444 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:46.235445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:46.235623 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.235712 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.235834 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.235908 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:57:46.237946 4211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:46.240015 4211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:46.247545 4744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.243369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38672 (local address 127.31.43.195:37581)
0812 03:57:46.243585 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.243590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.243741 (+ 151us) server_negotiation.cc:407] Connection header received
0812 03:57:46.243833 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.243835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.243852 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.243956 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:46.244180 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.245115 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.246454 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.247100 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.247190 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.247258 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.247392 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.247444 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":86,"threads_started":1}
I20260812 03:57:46.247611 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.242770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50268 (local address 127.31.43.194:40515)
0812 03:57:46.242942 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.242947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.242967 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:57:46.243004 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.243006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.243019 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.243169 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:46.243482 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.244643 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.246137 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.247134 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.247219 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.247281 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.247465 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.247511 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260812 03:57:46.247606 4737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.242634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:40515 (local address 127.0.0.1:50268)
0812 03:57:46.242801 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.242846 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.242916 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.243282 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.243285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.243287 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:46.243388 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.243390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.244769 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.244770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.246018 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.246021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.247314 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.247315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.247380 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.247383 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.247445 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.247481 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":111,"threads_started":1}
I20260812 03:57:46.247606 4739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.242913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45133 (local address 127.0.0.1:48660)
0812 03:57:46.243091 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.243112 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.243162 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.243585 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.243586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.243588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:46.243625 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.243626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.244945 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.244946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.246113 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.246117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.247253 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.247255 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.247331 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.247336 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.247372 (+ 36us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.247467 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":2,"thread_start_us":115,"threads_started":1}
I20260812 03:57:46.248334 4742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.243201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48660 (local address 127.31.43.193:45133)
0812 03:57:46.243398 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.243402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.243412 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:46.243453 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.243455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.243473 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.243697 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:46.243785 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.244833 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.246280 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.247134 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.247884 (+ 750us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.247971 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.248003 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.248238 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260812 03:57:46.248854 4501 tablet_service.cc:1467] Processing CreateTablet for tablet b115a963a23d48e9aacf10a5f4a05b8e (DEFAULT_TABLE table=test-table [id=78478fe996654ff6a7fd802bb2b4ce19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:46.249006 4352 tablet_service.cc:1467] Processing CreateTablet for tablet b115a963a23d48e9aacf10a5f4a05b8e (DEFAULT_TABLE table=test-table [id=78478fe996654ff6a7fd802bb2b4ce19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:46.249248 4501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b115a963a23d48e9aacf10a5f4a05b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:46.249414 4352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b115a963a23d48e9aacf10a5f4a05b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:46.251080 4740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.243130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:37581 (local address 127.0.0.1:38672)
0812 03:57:46.243679 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.243689 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.243747 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.243956 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.243957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.243959 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:46.244018 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.244020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.245233 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.245233 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.246327 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.246329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.247193 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.247194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.247279 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.247293 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.247369 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.247838 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":406,"thread_start_us":85,"threads_started":1}
I20260812 03:57:46.251893 4655 tablet_service.cc:1467] Processing CreateTablet for tablet b115a963a23d48e9aacf10a5f4a05b8e (DEFAULT_TABLE table=test-table [id=78478fe996654ff6a7fd802bb2b4ce19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:46.252277 4655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b115a963a23d48e9aacf10a5f4a05b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:46.252838 4746 tablet_bootstrap.cc:492] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5: Bootstrap starting.
I20260812 03:57:46.252942 4745 tablet_bootstrap.cc:492] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806: Bootstrap starting.
I20260812 03:57:46.253432 4746 tablet_bootstrap.cc:654] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:46.253654 4745 tablet_bootstrap.cc:654] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:46.254102 4746 log.cc:824] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:46.254385 4745 log.cc:824] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:46.255059 4746 tablet_bootstrap.cc:492] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5: No bootstrap required, opened a new log
I20260812 03:57:46.255141 4746 ts_tablet_manager.cc:1397] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:57:46.256078 4750 tablet_bootstrap.cc:492] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea: Bootstrap starting.
I20260812 03:57:46.256692 4750 tablet_bootstrap.cc:654] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:46.256695 4746 raft_consensus.cc:348] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ab76e815cd46faabb73959920bc806" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 40515 } } peers { permanent_uuid: "56e8135d0a1c46afb5134962759712e5" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45133 } } peers { permanent_uuid: "6edf733b72f344edb2f172eb020ff2ea" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37581 } }
I20260812 03:57:46.256901 4746 raft_consensus.cc:374] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:46.256949 4746 raft_consensus.cc:729] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e8135d0a1c46afb5134962759712e5, State: Initialized, Role: FOLLOWER
I20260812 03:57:46.257074 4746 consensus_queue.cc:260] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ab76e815cd46faabb73959920bc806" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 40515 } } peers { permanent_uuid: "56e8135d0a1c46afb5134962759712e5" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45133 } } peers { permanent_uuid: "6edf733b72f344edb2f172eb020ff2ea" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37581 } }
I20260812 03:57:46.257375 4746 ts_tablet_manager.cc:1428] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:57:46.257417 4422 heartbeater.cc:503] Master 127.31.43.254:43931 was elected leader, sending a full tablet report...
I20260812 03:57:46.257460 4750 log.cc:824] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:46.258142 4745 tablet_bootstrap.cc:492] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806: No bootstrap required, opened a new log
I20260812 03:57:46.258219 4750 tablet_bootstrap.cc:492] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea: No bootstrap required, opened a new log
I20260812 03:57:46.258224 4745 ts_tablet_manager.cc:1397] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 03:57:46.258278 4750 ts_tablet_manager.cc:1397] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:57:46.260557 4745 raft_consensus.cc:348] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ab76e815cd46faabb73959920bc806" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 40515 } } peers { permanent_uuid: "56e8135d0a1c46afb5134962759712e5" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45133 } } peers { permanent_uuid: "6edf733b72f344edb2f172eb020ff2ea" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37581 } }
I20260812 03:57:46.260557 4750 raft_consensus.cc:348] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ab76e815cd46faabb73959920bc806" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 40515 } } peers { permanent_uuid: "56e8135d0a1c46afb5134962759712e5" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45133 } } peers { permanent_uuid: "6edf733b72f344edb2f172eb020ff2ea" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37581 } }
I20260812 03:57:46.260773 4750 raft_consensus.cc:374] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:46.260773 4745 raft_consensus.cc:374] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:46.260814 4750 raft_consensus.cc:729] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6edf733b72f344edb2f172eb020ff2ea, State: Initialized, Role: FOLLOWER
I20260812 03:57:46.260814 4745 raft_consensus.cc:729] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34ab76e815cd46faabb73959920bc806, State: Initialized, Role: FOLLOWER
I20260812 03:57:46.260969 4750 consensus_queue.cc:260] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ab76e815cd46faabb73959920bc806" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 40515 } } peers { permanent_uuid: "56e8135d0a1c46afb5134962759712e5" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45133 } } peers { permanent_uuid: "6edf733b72f344edb2f172eb020ff2ea" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37581 } }
I20260812 03:57:46.260954 4745 consensus_queue.cc:260] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ab76e815cd46faabb73959920bc806" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 40515 } } peers { permanent_uuid: "56e8135d0a1c46afb5134962759712e5" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45133 } } peers { permanent_uuid: "6edf733b72f344edb2f172eb020ff2ea" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37581 } }
I20260812 03:57:46.261312 4745 ts_tablet_manager.cc:1428] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:57:46.261312 4750 ts_tablet_manager.cc:1428] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:57:46.261354 4572 heartbeater.cc:503] Master 127.31.43.254:43931 was elected leader, sending a full tablet report...
I20260812 03:57:46.261447 4726 heartbeater.cc:503] Master 127.31.43.254:43931 was elected leader, sending a full tablet report...
I20260812 03:57:46.289582 4751 raft_consensus.cc:482] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:46.289736 4751 raft_consensus.cc:504] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ab76e815cd46faabb73959920bc806" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 40515 } } peers { permanent_uuid: "56e8135d0a1c46afb5134962759712e5" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45133 } } peers { permanent_uuid: "6edf733b72f344edb2f172eb020ff2ea" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37581 } }
I20260812 03:57:46.290063 4751 leader_election.cc:288] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34ab76e815cd46faabb73959920bc806 (127.31.43.194:40515), 6edf733b72f344edb2f172eb020ff2ea (127.31.43.195:37581)
W20260812 03:57:46.293459 4573 tablet.cc:2369] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:57:46.294056 4424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.290330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:40515 (local address 127.31.43.193:40345)
0812 03:57:46.290406 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.290421 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.290502 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.290784 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.290786 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.290788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:46.290887 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.290890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.291739 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.291740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.292851 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.292854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.293729 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.293730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.293829 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.293833 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.293923 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.293975 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 03:57:46.294056 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.290498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:40345 (local address 127.31.43.194:40515)
0812 03:57:46.290575 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.290577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.290590 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:57:46.290633 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.290634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.290645 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.290761 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:46.291015 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.291618 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.292962 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.293652 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.293729 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.293740 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.293932 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.293975 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:57:46.294277 4744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.290714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:49831 (local address 127.31.43.195:37581)
0812 03:57:46.290770 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.290772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.290865 (+ 93us) server_negotiation.cc:407] Connection header received
0812 03:57:46.290917 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.290918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.290929 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.290991 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:46.291175 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.291736 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.293183 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.293827 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.293923 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.293932 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.294162 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.294208 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:57:46.294277 4755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.290542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:37581 (local address 127.31.43.193:49831)
0812 03:57:46.290767 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.290780 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.290847 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.291014 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.291015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.291017 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:46.291097 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.291099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.291894 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.291895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.293084 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:46.293087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.293939 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.293940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:46.294031 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.294084 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.294139 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.294187 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":159,"threads_started":1}
I20260812 03:57:46.294819 4675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b115a963a23d48e9aacf10a5f4a05b8e" candidate_uuid: "56e8135d0a1c46afb5134962759712e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6edf733b72f344edb2f172eb020ff2ea" is_pre_election: true
I20260812 03:57:46.294847 4523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b115a963a23d48e9aacf10a5f4a05b8e" candidate_uuid: "56e8135d0a1c46afb5134962759712e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34ab76e815cd46faabb73959920bc806" is_pre_election: true
I20260812 03:57:46.295042 4675 raft_consensus.cc:2454] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e8135d0a1c46afb5134962759712e5 in term 0.
I20260812 03:57:46.295053 4523 raft_consensus.cc:2454] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e8135d0a1c46afb5134962759712e5 in term 0.
I20260812 03:57:46.295375 4297 leader_election.cc:302] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34ab76e815cd46faabb73959920bc806, 56e8135d0a1c46afb5134962759712e5; no voters:
I20260812 03:57:46.295619 4751 raft_consensus.cc:2781] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:46.295734 4751 raft_consensus.cc:482] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:46.295771 4751 raft_consensus.cc:3037] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:46.296622 4751 raft_consensus.cc:504] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ab76e815cd46faabb73959920bc806" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 40515 } } peers { permanent_uuid: "56e8135d0a1c46afb5134962759712e5" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45133 } } peers { permanent_uuid: "6edf733b72f344edb2f172eb020ff2ea" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37581 } }
I20260812 03:57:46.296793 4751 leader_election.cc:288] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [CANDIDATE]: Term 1 election: Requested vote from peers 34ab76e815cd46faabb73959920bc806 (127.31.43.194:40515), 6edf733b72f344edb2f172eb020ff2ea (127.31.43.195:37581)
I20260812 03:57:46.297052 4675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b115a963a23d48e9aacf10a5f4a05b8e" candidate_uuid: "56e8135d0a1c46afb5134962759712e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6edf733b72f344edb2f172eb020ff2ea"
I20260812 03:57:46.297143 4675 raft_consensus.cc:3037] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:46.297196 4523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b115a963a23d48e9aacf10a5f4a05b8e" candidate_uuid: "56e8135d0a1c46afb5134962759712e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34ab76e815cd46faabb73959920bc806"
I20260812 03:57:46.297314 4523 raft_consensus.cc:3037] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:46.298026 4675 raft_consensus.cc:2454] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e8135d0a1c46afb5134962759712e5 in term 1.
I20260812 03:57:46.298218 4297 leader_election.cc:302] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56e8135d0a1c46afb5134962759712e5, 6edf733b72f344edb2f172eb020ff2ea; no voters:
I20260812 03:57:46.298331 4751 raft_consensus.cc:2781] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:46.298425 4523 raft_consensus.cc:2454] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e8135d0a1c46afb5134962759712e5 in term 1.
I20260812 03:57:46.298552 4751 raft_consensus.cc:686] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [term 1 LEADER]: Becoming Leader. State: Replica: 56e8135d0a1c46afb5134962759712e5, State: Running, Role: LEADER
I20260812 03:57:46.298792 4751 consensus_queue.cc:237] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ab76e815cd46faabb73959920bc806" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 40515 } } peers { permanent_uuid: "56e8135d0a1c46afb5134962759712e5" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45133 } } peers { permanent_uuid: "6edf733b72f344edb2f172eb020ff2ea" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37581 } }
I20260812 03:57:46.299769 4210 catalog_manager.cc:5495] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56e8135d0a1c46afb5134962759712e5 (127.31.43.193). New cstate: current_term: 1 leader_uuid: "56e8135d0a1c46afb5134962759712e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ab76e815cd46faabb73959920bc806" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 40515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e8135d0a1c46afb5134962759712e5" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6edf733b72f344edb2f172eb020ff2ea" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37581 } health_report { overall_health: UNKNOWN } } }
I20260812 03:57:46.309343 4742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.306709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.31.43.193:45133)
0812 03:57:46.306782 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.306786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.306801 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:57:46.306840 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.306841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.306858 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.306919 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:46.307144 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.307636 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.308426 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:46.308626 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:46.308681 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:46.308845 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:46.308955 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:46.309023 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.309173 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.309195 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:57:46.311947 4523 raft_consensus.cc:1261] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806 [term 1 FOLLOWER]: Refusing update from remote peer 56e8135d0a1c46afb5134962759712e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:46.312345 4675 raft_consensus.cc:1261] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea [term 1 FOLLOWER]: Refusing update from remote peer 56e8135d0a1c46afb5134962759712e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:46.312515 4756 consensus_queue.cc:1035] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34ab76e815cd46faabb73959920bc806" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 40515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:46.312700 4751 consensus_queue.cc:1035] T b115a963a23d48e9aacf10a5f4a05b8e P 56e8135d0a1c46afb5134962759712e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6edf733b72f344edb2f172eb020ff2ea" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:46.317924 4761 mvcc.cc:205] Tried to move back new op lower bound from 7317532943609450496 to 7317532943560544256. Current Snapshot: MvccSnapshot[applied={T|T < 7317532943609450496}]
I20260812 03:57:46.318349 4210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:46.318455 4210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:46.319294 4210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71c038842dbd4f4681abe94b6d54d4ff: Sending DeleteTablet for 3 replicas of tablet b115a963a23d48e9aacf10a5f4a05b8e
I20260812 03:57:46.320013 4501 tablet_service.cc:1514] Processing DeleteTablet for tablet b115a963a23d48e9aacf10a5f4a05b8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:46 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:50268
I20260812 03:57:46.320073 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 4278
I20260812 03:57:46.320257 4352 tablet_service.cc:1514] Processing DeleteTablet for tablet b115a963a23d48e9aacf10a5f4a05b8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:46 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:48660
I20260812 03:57:46.320706 4655 tablet_service.cc:1514] Processing DeleteTablet for tablet b115a963a23d48e9aacf10a5f4a05b8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:46 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:38672
I20260812 03:57:46.322639 4770 tablet_replica.cc:330] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806: stopping tablet replica
I20260812 03:57:46.322803 4770 raft_consensus.cc:2229] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:46.322971 4770 raft_consensus.cc:2258] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:46.323058 4771 tablet_replica.cc:330] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea: stopping tablet replica
I20260812 03:57:46.323261 4771 raft_consensus.cc:2229] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:46.323356 4771 raft_consensus.cc:2258] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:46.325428 4770 ts_tablet_manager.cc:1905] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:46.325546 4771 ts_tablet_manager.cc:1905] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:46.327410 4771 ts_tablet_manager.cc:1918] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:46.327411 4770 ts_tablet_manager.cc:1918] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:46.327520 4770 log.cc:1196] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/b115a963a23d48e9aacf10a5f4a05b8e
I20260812 03:57:46.327522 4771 log.cc:1196] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/b115a963a23d48e9aacf10a5f4a05b8e
I20260812 03:57:46.327896 4770 ts_tablet_manager.cc:1939] T b115a963a23d48e9aacf10a5f4a05b8e P 34ab76e815cd46faabb73959920bc806: Deleting consensus metadata
I20260812 03:57:46.327896 4771 ts_tablet_manager.cc:1939] T b115a963a23d48e9aacf10a5f4a05b8e P 6edf733b72f344edb2f172eb020ff2ea: Deleting consensus metadata
I20260812 03:57:46.328518 4194 catalog_manager.cc:4841] TS 6edf733b72f344edb2f172eb020ff2ea (127.31.43.195:37581): tablet b115a963a23d48e9aacf10a5f4a05b8e (table test-table [id=78478fe996654ff6a7fd802bb2b4ce19]) successfully deleted
I20260812 03:57:46.328648 4194 catalog_manager.cc:4841] TS 34ab76e815cd46faabb73959920bc806 (127.31.43.194:40515): tablet b115a963a23d48e9aacf10a5f4a05b8e (table test-table [id=78478fe996654ff6a7fd802bb2b4ce19]) successfully deleted
W20260812 03:57:46.332180 4194 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:45133 (error 108)
I20260812 03:57:46.332403 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 4428
I20260812 03:57:46.339754 4740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.339492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45133 (local address 127.0.0.1:48686)
0812 03:57:46.339624 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.339694 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:45133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260812 03:57:46.339936 4194 catalog_manager.cc:4584] TS 56e8135d0a1c46afb5134962759712e5 (127.31.43.193:45133): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b115a963a23d48e9aacf10a5f4a05b8e: Network error: Client connection negotiation failed: client connection to 127.31.43.193:45133: connect: Connection refused (error 111)
I20260812 03:57:46.345079 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 4577
I20260812 03:57:46.355682 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 4174
2026-08-12T03:57:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2179 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T03:57:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:46Z Disabled control of system clock
I20260812 03:57:46.389935 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40205
--webserver_interface=127.31.43.254
--webserver_port=0
--builtin_ntp_servers=127.31.43.212:37017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:40205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:57:46.477234 4776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:46.477480 4776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:46.477504 4776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:46.478663 4776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:46.478745 4776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:46.478760 4776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:46.478772 4776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:46.478785 4776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:46.480212 4776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:37017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:40205
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:46.480455 4776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:46.480726 4776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:46.486044 4784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:46.486028 4781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:46.486029 4782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:46.486330 4776 server_base.cc:1034] running on GCE node
I20260812 03:57:46.486739 4776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:46.487135 4776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:46.488312 4776 hybrid_clock.cc:648] HybridClock initialized: now 1786507066488283 us; error 47 us; skew 500 ppm
I20260812 03:57:46.489715 4776 webserver.cc:460] Webserver started at http://127.31.43.254:45261/ using document root <none> and password file <none>
I20260812 03:57:46.490020 4776 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:46.490074 4776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:46.490257 4776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:46.491479 4776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "e3eff8a61b2a4756ab37c27d8fb21066"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "b478ff01dd17cc1014b5b3c2bd68fe9b"
server_key_iv: "bc17df320d03543cb806a1ae0d2addb9"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.491869 4776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "e3eff8a61b2a4756ab37c27d8fb21066"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "b478ff01dd17cc1014b5b3c2bd68fe9b"
server_key_iv: "bc17df320d03543cb806a1ae0d2addb9"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.493379 4776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:57:46.494413 4793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.494745 4776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:46.494859 4776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "e3eff8a61b2a4756ab37c27d8fb21066"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "b478ff01dd17cc1014b5b3c2bd68fe9b"
server_key_iv: "bc17df320d03543cb806a1ae0d2addb9"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.494967 4776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:46.498600 4776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:46.498939 4776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:46.499055 4776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:46.503610 4776 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:40205
I20260812 03:57:46.503665 4851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:40205 every 8 connection(s)
I20260812 03:57:46.504087 4776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:46.504858 4852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:46.505417 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 4776
I20260812 03:57:46.505575 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:46.505774 31919 external_mini_cluster.cc:1428] Setting key 9e52d52bf73de63a3e9f99e89742d4b1
I20260812 03:57:46.508781 4855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.506430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49150 (local address 127.31.43.254:40205)
0812 03:57:46.506727 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.506738 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.506780 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:57:46.507296 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.507304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.507350 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.507478 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.508251 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.508255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.508255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:46.508260 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.508304 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:46.508315 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.508376 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.508483 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.508563 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":110,"threads_started":1}
I20260812 03:57:46.509440 4852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066: Bootstrap starting.
I20260812 03:57:46.510149 4852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:46.510409 4852 log.cc:824] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:46.511371 4852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066: No bootstrap required, opened a new log
I20260812 03:57:46.512750 4852 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3eff8a61b2a4756ab37c27d8fb21066" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40205 } }
I20260812 03:57:46.512902 4852 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:46.512926 4852 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3eff8a61b2a4756ab37c27d8fb21066, State: Initialized, Role: FOLLOWER
I20260812 03:57:46.513018 4852 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3eff8a61b2a4756ab37c27d8fb21066" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40205 } }
I20260812 03:57:46.513082 4852 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:46.513104 4852 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:46.513149 4852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:46.513859 4852 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3eff8a61b2a4756ab37c27d8fb21066" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40205 } }
I20260812 03:57:46.513962 4852 leader_election.cc:302] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3eff8a61b2a4756ab37c27d8fb21066; no voters:
I20260812 03:57:46.514101 4852 leader_election.cc:288] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:46.514385 4852 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:46.514519 4857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:46.514747 4857 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [term 1 LEADER]: Becoming Leader. State: Replica: e3eff8a61b2a4756ab37c27d8fb21066, State: Running, Role: LEADER
I20260812 03:57:46.514891 4857 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3eff8a61b2a4756ab37c27d8fb21066" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40205 } }
I20260812 03:57:46.515407 4859 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3eff8a61b2a4756ab37c27d8fb21066. Latest consensus state: current_term: 1 leader_uuid: "e3eff8a61b2a4756ab37c27d8fb21066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3eff8a61b2a4756ab37c27d8fb21066" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40205 } } }
I20260812 03:57:46.515537 4859 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:46.515565 4858 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3eff8a61b2a4756ab37c27d8fb21066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3eff8a61b2a4756ab37c27d8fb21066" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40205 } } }
I20260812 03:57:46.515668 4858 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:46.515935 4869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:46.516438 4869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:46.518121 4869 catalog_manager.cc:1347] Generated new cluster ID: 87be362b78de4d14813e70757b8bdd85
I20260812 03:57:46.518177 4869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:46.534451 4869 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:46.534694 4869 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:46.538854 4869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066: Generated new TSK 0
I20260812 03:57:46.539093 4869 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:57:46.541142 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40205
--builtin_ntp_servers=127.31.43.212:37017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:57:46.636794 4878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:46.637090 4878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:46.637135 4878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:46.638447 4878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:46.638533 4878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:46.638586 4878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:46.640164 4878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:37017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:46.640478 4878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:46.640836 4878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:46.646641 4887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:46.646646 4883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:46.646970 4878 server_base.cc:1034] running on GCE node
W20260812 03:57:46.646641 4885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:46.647294 4878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:46.647622 4878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:46.648819 4878 hybrid_clock.cc:648] HybridClock initialized: now 1786507066648799 us; error 36 us; skew 500 ppm
I20260812 03:57:46.650393 4878 webserver.cc:460] Webserver started at http://127.31.43.193:34489/ using document root <none> and password file <none>
I20260812 03:57:46.650790 4878 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:46.650875 4878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:46.651122 4878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:46.652318 4878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "208a1aaf53fc4e19a0303f00778cf3a8"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "59cab069af1a7c89816f626f507c9f93"
server_key_iv: "31efe3679e318e526285a4cd132f804f"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.652719 4878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "208a1aaf53fc4e19a0303f00778cf3a8"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "59cab069af1a7c89816f626f507c9f93"
server_key_iv: "31efe3679e318e526285a4cd132f804f"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.654492 4878 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 03:57:46.655543 4896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.655808 4878 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:46.655908 4878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "208a1aaf53fc4e19a0303f00778cf3a8"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "59cab069af1a7c89816f626f507c9f93"
server_key_iv: "31efe3679e318e526285a4cd132f804f"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.655977 4878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:46.659899 4878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:46.660250 4878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:46.660377 4878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:46.660574 4878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:46.660877 4878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:46.660912 4878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.660939 4878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:46.660953 4878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.668226 4878 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:41763
I20260812 03:57:46.668277 5021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:41763 every 8 connection(s)
I20260812 03:57:46.668725 4878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:57:46.672349 4855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.669684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:45497 (local address 127.31.43.254:40205)
0812 03:57:46.669779 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.669781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.670068 (+ 287us) server_negotiation.cc:407] Connection header received
0812 03:57:46.670712 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.670714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.670721 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.670770 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.672062 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.672063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.672064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:46.672068 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.672086 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:46.672090 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.672147 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.672254 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.672271 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:57:46.672598 5025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.669674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40205 (local address 127.31.43.193:45497)
0812 03:57:46.669921 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.669959 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.670554 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.670863 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.670867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.670875 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:46.671880 (+ 1005us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:57:46.671881 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:46.671911 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:57:46.671946 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:46.672152 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:46.672158 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:46.672161 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.672252 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.672322 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":96,"threads_started":1}
I20260812 03:57:46.673210 5022 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40205
I20260812 03:57:46.673303 5022 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:46.673493 5022 heartbeater.cc:511] Master 127.31.43.254:40205 requested a full tablet report, sending...
I20260812 03:57:46.674082 4811 ts_manager.cc:194] Registered new tserver with Master: 208a1aaf53fc4e19a0303f00778cf3a8 (127.31.43.193:41763)
I20260812 03:57:46.677760 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 4878
I20260812 03:57:46.677858 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:57:46.678117 31919 external_mini_cluster.cc:1428] Setting key 73e09a43853056a3ab4548457a56b5b9
I20260812 03:57:46.679282 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40205
--builtin_ntp_servers=127.31.43.212:37017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:57:46.767724 5026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:46.767970 5026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:46.767998 5026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:46.769387 5026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:46.769447 5026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:46.769479 5026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:46.771051 5026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:37017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:46.771350 5026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:46.771699 5026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:46.777642 5034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:46.777598 5036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:46.777590 5033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:46.778003 5026 server_base.cc:1034] running on GCE node
I20260812 03:57:46.778240 5026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:46.778507 5026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:46.779682 5026 hybrid_clock.cc:648] HybridClock initialized: now 1786507066779656 us; error 40 us; skew 500 ppm
I20260812 03:57:46.781159 5026 webserver.cc:460] Webserver started at http://127.31.43.194:45153/ using document root <none> and password file <none>
I20260812 03:57:46.781474 5026 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:46.781549 5026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:46.781728 5026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:46.783004 5026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "d5b0d932cd534235a366506a84d7e765"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "4d1c70793af82c425423a8cef90946df"
server_key_iv: "0fe1ec0270d3b8046a7b9866db240675"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.783407 5026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "d5b0d932cd534235a366506a84d7e765"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "4d1c70793af82c425423a8cef90946df"
server_key_iv: "0fe1ec0270d3b8046a7b9866db240675"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.785015 5026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:57:46.786101 5043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.786379 5026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:46.786484 5026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "d5b0d932cd534235a366506a84d7e765"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "4d1c70793af82c425423a8cef90946df"
server_key_iv: "0fe1ec0270d3b8046a7b9866db240675"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.786550 5026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:46.790452 5026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:46.790815 5026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:46.790941 5026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:46.791145 5026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:46.791450 5026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:46.791482 5026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.791505 5026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:46.791519 5026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.798914 5026 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:32771
I20260812 03:57:46.798956 5172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:32771 every 8 connection(s)
I20260812 03:57:46.799413 5026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:57:46.803439 4855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.800536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:44011 (local address 127.31.43.254:40205)
0812 03:57:46.800603 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.800605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.800792 (+ 187us) server_negotiation.cc:407] Connection header received
0812 03:57:46.801328 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.801329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.801335 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.801408 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.803038 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.803040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.803040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:46.803045 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.803062 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:46.803067 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.803148 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.803298 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.803326 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:57:46.803568 5175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.800442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40205 (local address 127.31.43.194:44011)
0812 03:57:46.800647 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.800692 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.801210 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.801664 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.801668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.801678 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:46.802849 (+ 1171us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:57:46.802849 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:46.802891 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:57:46.802927 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:46.803168 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:46.803175 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:46.803178 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.803273 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.803339 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
I20260812 03:57:46.804286 5173 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40205
I20260812 03:57:46.804427 5173 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:46.804632 5173 heartbeater.cc:511] Master 127.31.43.254:40205 requested a full tablet report, sending...
I20260812 03:57:46.805075 4811 ts_manager.cc:194] Registered new tserver with Master: d5b0d932cd534235a366506a84d7e765 (127.31.43.194:32771)
I20260812 03:57:46.805783 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 5026
I20260812 03:57:46.805936 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:57:46.806223 31919 external_mini_cluster.cc:1428] Setting key 67365a5310d206687e0982e4d3236cf5
I20260812 03:57:46.807476 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40205
--builtin_ntp_servers=127.31.43.212:37017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:57:46.906134 5176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:46.906383 5176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:46.906417 5176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:46.907850 5176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:46.907902 5176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:46.907934 5176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:46.909843 5176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:37017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:46.910167 5176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:46.910575 5176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:46.916638 5184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:46.916638 5182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:46.916679 5181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:46.916957 5176 server_base.cc:1034] running on GCE node
I20260812 03:57:46.917246 5176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:46.917534 5176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:46.918740 5176 hybrid_clock.cc:648] HybridClock initialized: now 1786507066918716 us; error 32 us; skew 500 ppm
I20260812 03:57:46.920303 5176 webserver.cc:460] Webserver started at http://127.31.43.195:33719/ using document root <none> and password file <none>
I20260812 03:57:46.920632 5176 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:46.920691 5176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:46.920845 5176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:46.922099 5176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "8e1f2ed711594f76843dcd0eac1bb6fe"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "37ef6fc51200c93ceffbe2e48d84fb34"
server_key_iv: "de1655d03a426cfdbe1c2b1aece9fe63"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.922506 5176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "8e1f2ed711594f76843dcd0eac1bb6fe"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "37ef6fc51200c93ceffbe2e48d84fb34"
server_key_iv: "de1655d03a426cfdbe1c2b1aece9fe63"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.924177 5176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:46.925048 5190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.925336 5176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:46.925443 5176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "8e1f2ed711594f76843dcd0eac1bb6fe"
format_stamp: "Formatted at 2026-08-12 03:57:46 on dist-test-slave-8tdl"
server_key: "37ef6fc51200c93ceffbe2e48d84fb34"
server_key_iv: "de1655d03a426cfdbe1c2b1aece9fe63"
server_key_version: "encryptionkey@0"
I20260812 03:57:46.925524 5176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:46.929322 5176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:46.929629 5176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:46.929772 5176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:46.930042 5176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:46.930371 5176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:46.930405 5176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.930426 5176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:46.930439 5176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:46.936664 5176 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:37345
I20260812 03:57:46.936713 5325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:37345 every 8 connection(s)
I20260812 03:57:46.937090 5176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:57:46.940747 4855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.937959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:33675 (local address 127.31.43.254:40205)
0812 03:57:46.938040 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.938043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.938250 (+ 207us) server_negotiation.cc:407] Connection header received
0812 03:57:46.938858 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.938859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.938866 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.938940 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.940391 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.940393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.940393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:46.940398 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.940421 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:46.940430 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.940498 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.940663 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.940686 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:57:46.941023 5329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.937900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40205 (local address 127.31.43.195:33675)
0812 03:57:46.938115 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.938161 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.938640 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.939006 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.939009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.939018 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:46.940141 (+ 1123us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:57:46.940142 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:46.940177 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:57:46.940211 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:46.940531 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:46.940537 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:46.940540 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.940633 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.940693 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
I20260812 03:57:46.941634 5326 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40205
I20260812 03:57:46.941758 5326 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:46.941936 5326 heartbeater.cc:511] Master 127.31.43.254:40205 requested a full tablet report, sending...
I20260812 03:57:46.942584 4811 ts_manager.cc:194] Registered new tserver with Master: 8e1f2ed711594f76843dcd0eac1bb6fe (127.31.43.195:37345)
I20260812 03:57:46.944279 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 5176
I20260812 03:57:46.944376 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:57:46.944595 31919 external_mini_cluster.cc:1428] Setting key 1dc545ef382ae316c5d1c8cea7aed11e
I20260812 03:57:46.945923 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:57:46.949028 4855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.947727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49164 (local address 127.31.43.254:40205)
0812 03:57:46.947793 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.947795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.947883 (+ 88us) server_negotiation.cc:407] Connection header received
0812 03:57:46.947936 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.947937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.947942 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.948002 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.948782 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.948784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.948784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:46.948789 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.948805 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:46.948809 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.948863 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.948958 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.948971 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:57:46.951191 4811 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:46.953032 4811 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:46.958186 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.956115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:41763 (local address 127.0.0.1:53124)
0812 03:57:46.956366 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.956421 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.956491 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.956820 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.956828 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.956836 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:46.957492 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:57:46.957493 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:46.957513 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:57:46.957545 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:46.958025 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:46.958027 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:46.958027 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.958087 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.958105 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"spinlock_wait_cycles":896,"thread_start_us":135,"threads_started":1}
I20260812 03:57:46.958240 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.956205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53124 (local address 127.31.43.193:41763)
0812 03:57:46.956667 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.956671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.956684 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:57:46.956715 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.956717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.956726 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.956817 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.957842 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.957854 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.957854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:46.957859 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.957895 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:46.957935 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.958009 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.958106 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.958125 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":89,"threads_started":1}
I20260812 03:57:46.958375 5341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.956414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59160 (local address 127.31.43.194:32771)
0812 03:57:46.956666 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.956671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.956684 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:57:46.956715 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.956717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.956726 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.956817 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.957776 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.957786 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.957787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:46.957793 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.957836 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:46.957866 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.957929 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.958232 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.958251 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":105,"threads_started":1}
I20260812 03:57:46.958487 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.956414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50302 (local address 127.31.43.195:37345)
0812 03:57:46.957217 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.957221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.957349 (+ 128us) server_negotiation.cc:407] Connection header received
0812 03:57:46.957411 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.957414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.957423 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.957500 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.958033 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.958041 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.958041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:46.958045 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.958079 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:46.958106 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.958166 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.958391 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.958407 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":766,"thread_start_us":111,"threads_started":1}
I20260812 03:57:46.958189 5336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.956117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:32771 (local address 127.0.0.1:59160)
0812 03:57:46.956426 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.956437 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.956491 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.957148 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.957149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.957152 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:46.957668 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:57:46.957669 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:46.957675 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:57:46.957685 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:46.957957 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:46.957964 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:46.957967 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.958050 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.958067 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":94,"threads_started":1}
I20260812 03:57:46.958334 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.956158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:37345 (local address 127.0.0.1:50302)
0812 03:57:46.957269 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.957278 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.957354 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.957504 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.957505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.957507 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:46.957926 (+ 419us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:57:46.957927 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:46.957934 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:57:46.957947 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:46.958192 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:46.958193 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:46.958194 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.958239 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.958253 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"thread_start_us":95,"threads_started":1}
I20260812 03:57:46.959435 5249 tablet_service.cc:1467] Processing CreateTablet for tablet 2023e61cf66540b997eef8545346e441 (DEFAULT_TABLE table=test-table [id=8f7d5eafe22643a1991b04eba9da87c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:46.959434 5094 tablet_service.cc:1467] Processing CreateTablet for tablet 2023e61cf66540b997eef8545346e441 (DEFAULT_TABLE table=test-table [id=8f7d5eafe22643a1991b04eba9da87c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:46.959882 5249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2023e61cf66540b997eef8545346e441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:46.959913 5094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2023e61cf66540b997eef8545346e441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:46.959832 4948 tablet_service.cc:1467] Processing CreateTablet for tablet 2023e61cf66540b997eef8545346e441 (DEFAULT_TABLE table=test-table [id=8f7d5eafe22643a1991b04eba9da87c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:46.960251 4948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2023e61cf66540b997eef8545346e441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:46.963632 5346 tablet_bootstrap.cc:492] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe: Bootstrap starting.
I20260812 03:57:46.964422 5346 tablet_bootstrap.cc:654] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:46.964798 5346 log.cc:824] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:46.965195 5343 tablet_bootstrap.cc:492] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8: Bootstrap starting.
I20260812 03:57:46.965840 5346 tablet_bootstrap.cc:492] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe: No bootstrap required, opened a new log
I20260812 03:57:46.965965 5346 ts_tablet_manager.cc:1397] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:57:46.966056 5343 tablet_bootstrap.cc:654] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:46.966419 5343 log.cc:824] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:46.966902 5345 tablet_bootstrap.cc:492] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765: Bootstrap starting.
I20260812 03:57:46.967415 5343 tablet_bootstrap.cc:492] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8: No bootstrap required, opened a new log
I20260812 03:57:46.967525 5343 ts_tablet_manager.cc:1397] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:57:46.967669 5345 tablet_bootstrap.cc:654] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:46.968093 5345 log.cc:824] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:46.969177 5345 tablet_bootstrap.cc:492] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765: No bootstrap required, opened a new log
I20260812 03:57:46.969262 5345 ts_tablet_manager.cc:1397] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:57:46.969273 5346 raft_consensus.cc:348] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208a1aaf53fc4e19a0303f00778cf3a8" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 41763 } } peers { permanent_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37345 } } peers { permanent_uuid: "d5b0d932cd534235a366506a84d7e765" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 32771 } }
I20260812 03:57:46.969537 5346 raft_consensus.cc:374] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:46.969585 5346 raft_consensus.cc:729] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e1f2ed711594f76843dcd0eac1bb6fe, State: Initialized, Role: FOLLOWER
I20260812 03:57:46.969780 5346 consensus_queue.cc:260] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208a1aaf53fc4e19a0303f00778cf3a8" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 41763 } } peers { permanent_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37345 } } peers { permanent_uuid: "d5b0d932cd534235a366506a84d7e765" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 32771 } }
I20260812 03:57:46.970130 5346 ts_tablet_manager.cc:1428] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 03:57:46.970258 5326 heartbeater.cc:503] Master 127.31.43.254:40205 was elected leader, sending a full tablet report...
I20260812 03:57:46.970705 5343 raft_consensus.cc:348] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208a1aaf53fc4e19a0303f00778cf3a8" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 41763 } } peers { permanent_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37345 } } peers { permanent_uuid: "d5b0d932cd534235a366506a84d7e765" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 32771 } }
I20260812 03:57:46.970904 5343 raft_consensus.cc:374] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:46.970956 5343 raft_consensus.cc:729] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 208a1aaf53fc4e19a0303f00778cf3a8, State: Initialized, Role: FOLLOWER
I20260812 03:57:46.971101 5343 consensus_queue.cc:260] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208a1aaf53fc4e19a0303f00778cf3a8" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 41763 } } peers { permanent_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37345 } } peers { permanent_uuid: "d5b0d932cd534235a366506a84d7e765" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 32771 } }
I20260812 03:57:46.971460 5343 ts_tablet_manager.cc:1428] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260812 03:57:46.971621 5022 heartbeater.cc:503] Master 127.31.43.254:40205 was elected leader, sending a full tablet report...
I20260812 03:57:46.972406 5345 raft_consensus.cc:348] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208a1aaf53fc4e19a0303f00778cf3a8" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 41763 } } peers { permanent_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37345 } } peers { permanent_uuid: "d5b0d932cd534235a366506a84d7e765" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 32771 } }
I20260812 03:57:46.972613 5345 raft_consensus.cc:374] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:46.972662 5345 raft_consensus.cc:729] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5b0d932cd534235a366506a84d7e765, State: Initialized, Role: FOLLOWER
I20260812 03:57:46.972868 5345 consensus_queue.cc:260] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208a1aaf53fc4e19a0303f00778cf3a8" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 41763 } } peers { permanent_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37345 } } peers { permanent_uuid: "d5b0d932cd534235a366506a84d7e765" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 32771 } }
I20260812 03:57:46.972983 5352 raft_consensus.cc:482] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:46.973094 5352 raft_consensus.cc:504] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208a1aaf53fc4e19a0303f00778cf3a8" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 41763 } } peers { permanent_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37345 } } peers { permanent_uuid: "d5b0d932cd534235a366506a84d7e765" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 32771 } }
I20260812 03:57:46.973134 5345 ts_tablet_manager.cc:1428] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260812 03:57:46.973222 5173 heartbeater.cc:503] Master 127.31.43.254:40205 was elected leader, sending a full tablet report...
I20260812 03:57:46.973875 5352 leader_election.cc:288] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 208a1aaf53fc4e19a0303f00778cf3a8 (127.31.43.193:41763), 8e1f2ed711594f76843dcd0eac1bb6fe (127.31.43.195:37345)
I20260812 03:57:46.975104 5175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.973650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:41763 (local address 127.31.43.194:47283)
0812 03:57:46.973706 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.973733 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.973772 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.974025 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.974027 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.974030 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:46.974699 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:57:46.974700 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:46.974710 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:57:46.974726 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:46.974912 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:46.974913 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:46.974914 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.974992 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.975009 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 03:57:46.975119 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.973708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:47283 (local address 127.31.43.193:41763)
0812 03:57:46.973755 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.973757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.973767 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:46.973863 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.973865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.973870 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.973932 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.974799 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.974800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.974800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:46.974805 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.974823 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:46.974828 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.974896 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.974994 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.975012 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:57:46.975471 4973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2023e61cf66540b997eef8545346e441" candidate_uuid: "d5b0d932cd534235a366506a84d7e765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "208a1aaf53fc4e19a0303f00778cf3a8" is_pre_election: true
I20260812 03:57:46.975646 4973 raft_consensus.cc:2454] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5b0d932cd534235a366506a84d7e765 in term 0.
I20260812 03:57:46.975915 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.974142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:58823 (local address 127.31.43.195:37345)
0812 03:57:46.974187 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.974189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.974560 (+ 371us) server_negotiation.cc:407] Connection header received
0812 03:57:46.974641 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.974642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.974648 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.974710 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.975632 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.975633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.975634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:46.975639 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.975658 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:46.975662 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.975729 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.975832 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.975848 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:57:46.975911 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.974096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:37345 (local address 127.31.43.194:58823)
0812 03:57:46.974500 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:46.974511 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:46.974563 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:46.974833 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:46.974834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:46.974835 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:46.975495 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:57:46.975496 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:46.975504 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:57:46.975518 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:46.975727 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:46.975728 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:46.975729 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:57:46.975823 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 03:57:46.975838 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":90,"threads_started":1}
I20260812 03:57:46.975898 5044 leader_election.cc:302] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 208a1aaf53fc4e19a0303f00778cf3a8, d5b0d932cd534235a366506a84d7e765; no voters:
I20260812 03:57:46.976248 5352 raft_consensus.cc:2781] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:46.976349 5352 raft_consensus.cc:482] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:46.976323 5277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2023e61cf66540b997eef8545346e441" candidate_uuid: "d5b0d932cd534235a366506a84d7e765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe" is_pre_election: true
I20260812 03:57:46.976388 5352 raft_consensus.cc:3037] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:46.976431 5277 raft_consensus.cc:2454] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5b0d932cd534235a366506a84d7e765 in term 0.
I20260812 03:57:46.977205 5352 raft_consensus.cc:504] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208a1aaf53fc4e19a0303f00778cf3a8" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 41763 } } peers { permanent_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37345 } } peers { permanent_uuid: "d5b0d932cd534235a366506a84d7e765" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 32771 } }
I20260812 03:57:46.977376 5352 leader_election.cc:288] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [CANDIDATE]: Term 1 election: Requested vote from peers 208a1aaf53fc4e19a0303f00778cf3a8 (127.31.43.193:41763), 8e1f2ed711594f76843dcd0eac1bb6fe (127.31.43.195:37345)
I20260812 03:57:46.977533 4973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2023e61cf66540b997eef8545346e441" candidate_uuid: "d5b0d932cd534235a366506a84d7e765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "208a1aaf53fc4e19a0303f00778cf3a8"
I20260812 03:57:46.977591 5277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2023e61cf66540b997eef8545346e441" candidate_uuid: "d5b0d932cd534235a366506a84d7e765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe"
I20260812 03:57:46.977669 5277 raft_consensus.cc:3037] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:46.977671 4973 raft_consensus.cc:3037] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:46.978461 5277 raft_consensus.cc:2454] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5b0d932cd534235a366506a84d7e765 in term 1.
I20260812 03:57:46.978466 4973 raft_consensus.cc:2454] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5b0d932cd534235a366506a84d7e765 in term 1.
I20260812 03:57:46.978657 5045 leader_election.cc:302] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e1f2ed711594f76843dcd0eac1bb6fe, d5b0d932cd534235a366506a84d7e765; no voters:
I20260812 03:57:46.978794 5352 raft_consensus.cc:2781] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:46.978855 5352 raft_consensus.cc:686] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 1 LEADER]: Becoming Leader. State: Replica: d5b0d932cd534235a366506a84d7e765, State: Running, Role: LEADER
I20260812 03:57:46.978945 5352 consensus_queue.cc:237] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208a1aaf53fc4e19a0303f00778cf3a8" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 41763 } } peers { permanent_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37345 } } peers { permanent_uuid: "d5b0d932cd534235a366506a84d7e765" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 32771 } }
I20260812 03:57:46.979882 4811 catalog_manager.cc:5495] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5b0d932cd534235a366506a84d7e765 (127.31.43.194). New cstate: current_term: 1 leader_uuid: "d5b0d932cd534235a366506a84d7e765" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208a1aaf53fc4e19a0303f00778cf3a8" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 41763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5b0d932cd534235a366506a84d7e765" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 32771 } health_report { overall_health: HEALTHY } } }
I20260812 03:57:46.990477 5341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:46.989241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59176 (local address 127.31.43.194:32771)
0812 03:57:46.989376 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:46.989378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:46.989390 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:57:46.989445 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:46.989446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:46.989462 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:46.989523 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:46.990165 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:46.990167 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:46.990167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:46.990172 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:46.990189 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:46.990193 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:46.990245 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:46.990361 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 03:57:46.990374 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 03:57:46.992671 5277 raft_consensus.cc:1261] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe [term 1 FOLLOWER]: Refusing update from remote peer d5b0d932cd534235a366506a84d7e765: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:46.992671 4973 raft_consensus.cc:1261] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8 [term 1 FOLLOWER]: Refusing update from remote peer d5b0d932cd534235a366506a84d7e765: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:46.993202 5353 consensus_queue.cc:1035] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e1f2ed711594f76843dcd0eac1bb6fe" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 37345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:46.993379 5352 consensus_queue.cc:1035] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [LEADER]: Connected to new peer: Peer: permanent_uuid: "208a1aaf53fc4e19a0303f00778cf3a8" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 41763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:46.996387 5357 mvcc.cc:205] Tried to move back new op lower bound from 7317532946398740480 to 7317532946346696704. Current Snapshot: MvccSnapshot[applied={T|T < 7317532946398740480}]
I20260812 03:57:46.996579 5359 mvcc.cc:205] Tried to move back new op lower bound from 7317532946398740480 to 7317532946346696704. Current Snapshot: MvccSnapshot[applied={T|T < 7317532946398740480}]
I20260812 03:57:46.997362 5358 mvcc.cc:205] Tried to move back new op lower bound from 7317532946398740480 to 7317532946346696704. Current Snapshot: MvccSnapshot[applied={T|T < 7317532946398740480}]
I20260812 03:57:46.999292 4810 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:46.999418 4810 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:47.000348 4810 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e3eff8a61b2a4756ab37c27d8fb21066: Sending DeleteTablet for 3 replicas of tablet 2023e61cf66540b997eef8545346e441
I20260812 03:57:47.000774 4948 tablet_service.cc:1514] Processing DeleteTablet for tablet 2023e61cf66540b997eef8545346e441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:46 UTC) from {username='slave'} at 127.0.0.1:53124
I20260812 03:57:47.000774 5094 tablet_service.cc:1514] Processing DeleteTablet for tablet 2023e61cf66540b997eef8545346e441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:46 UTC) from {username='slave'} at 127.0.0.1:59160
I20260812 03:57:47.001094 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 4878
I20260812 03:57:47.001180 5367 tablet_replica.cc:330] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8: stopping tablet replica
I20260812 03:57:47.001303 5367 raft_consensus.cc:2229] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:47.001431 5367 raft_consensus.cc:2258] T 2023e61cf66540b997eef8545346e441 P 208a1aaf53fc4e19a0303f00778cf3a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:47.002801 5368 tablet_replica.cc:330] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765: stopping tablet replica
I20260812 03:57:47.002923 5368 raft_consensus.cc:2229] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:57:47.003033 5368 raft_consensus.cc:2258] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:47.003528 5368 ts_tablet_manager.cc:1905] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:47.006016 5368 ts_tablet_manager.cc:1918] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:47.006129 5368 log.cc:1196] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/2023e61cf66540b997eef8545346e441
I20260812 03:57:47.006443 5368 ts_tablet_manager.cc:1939] T 2023e61cf66540b997eef8545346e441 P d5b0d932cd534235a366506a84d7e765: Deleting consensus metadata
I20260812 03:57:47.008080 5249 tablet_service.cc:1514] Processing DeleteTablet for tablet 2023e61cf66540b997eef8545346e441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:46 UTC) from {username='slave'} at 127.0.0.1:50302
I20260812 03:57:47.008193 4797 catalog_manager.cc:4841] TS d5b0d932cd534235a366506a84d7e765 (127.31.43.194:32771): tablet 2023e61cf66540b997eef8545346e441 (table test-table [id=8f7d5eafe22643a1991b04eba9da87c2]) successfully deleted
I20260812 03:57:47.008419 5369 tablet_replica.cc:330] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe: stopping tablet replica
I20260812 03:57:47.008558 5369 raft_consensus.cc:2229] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:47.008709 5369 raft_consensus.cc:2258] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:47.009239 5369 ts_tablet_manager.cc:1905] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:47.011437 5369 ts_tablet_manager.cc:1918] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260812 03:57:47.011618 4794 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:41763 (error 108)
I20260812 03:57:47.011737 5369 log.cc:1196] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/2023e61cf66540b997eef8545346e441
I20260812 03:57:47.011911 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 5026
I20260812 03:57:47.012065 5369 ts_tablet_manager.cc:1939] T 2023e61cf66540b997eef8545346e441 P 8e1f2ed711594f76843dcd0eac1bb6fe: Deleting consensus metadata
I20260812 03:57:47.013434 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:47.012592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:41763 (local address 127.0.0.1:53140)
0812 03:57:47.013322 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:47.013382 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":698}
W20260812 03:57:47.013677 4794 catalog_manager.cc:4584] TS 208a1aaf53fc4e19a0303f00778cf3a8 (127.31.43.193:41763): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2023e61cf66540b997eef8545346e441: Network error: Client connection negotiation failed: client connection to 127.31.43.193:41763: connect: Connection refused (error 111)
I20260812 03:57:47.014294 4795 catalog_manager.cc:4841] TS 8e1f2ed711594f76843dcd0eac1bb6fe (127.31.43.195:37345): tablet 2023e61cf66540b997eef8545346e441 (table test-table [id=8f7d5eafe22643a1991b04eba9da87c2]) successfully deleted
I20260812 03:57:47.020687 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 5176
I20260812 03:57:47.030393 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 4776
2026-08-12T03:57:47Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (676 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 03:57:47.057950 31919 mini_oidc.cc:152] Starting JWKS server
I20260812 03:57:47.059319 31919 webserver.cc:460] Webserver started at https://127.0.0.1:34443/ using document root <none> and password file <none>
I20260812 03:57:47.059413 31919 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:57:47.059557 31919 webserver.cc:460] Webserver started at http://127.0.0.1:33329/ using document root <none> and password file <none>
2026-08-12T03:57:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:47Z Disabled control of system clock
I20260812 03:57:47.070751 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40827
--webserver_interface=127.31.43.254
--webserver_port=0
--builtin_ntp_servers=127.31.43.212:46581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:40827
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34443/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/testchainca.cert with env {}
W20260812 03:57:47.162330 5378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:47.162585 5378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:47.162611 5378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:47.163915 5378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:47.163975 5378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:47.163988 5378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:47.164001 5378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:47.164014 5378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:57:47.164027 5378 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:57:47.164038 5378 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34443/jwks.json
I20260812 03:57:47.165828 5378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:46581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:40827
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34443/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:47.166231 5378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:47.166507 5378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:47.171929 5384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:47.171929 5383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:47.172205 5378 server_base.cc:1034] running on GCE node
W20260812 03:57:47.171927 5386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:47.172595 5378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:47.172948 5378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:47.174131 5378 hybrid_clock.cc:648] HybridClock initialized: now 1786507067174099 us; error 86 us; skew 500 ppm
I20260812 03:57:47.175731 5378 webserver.cc:460] Webserver started at http://127.31.43.254:43531/ using document root <none> and password file <none>
I20260812 03:57:47.176045 5378 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:47.176131 5378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:47.176307 5378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:47.177591 5378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "5198357f2d5c4ae8b1bcf6f997c6b7ba"
format_stamp: "Formatted at 2026-08-12 03:57:47 on dist-test-slave-8tdl"
server_key: "9d1b2fcc0add5a5ff7e890cd4501d187"
server_key_iv: "47cf33f11e544cb3a89e036e99b889de"
server_key_version: "encryptionkey@0"
I20260812 03:57:47.178071 5378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "5198357f2d5c4ae8b1bcf6f997c6b7ba"
format_stamp: "Formatted at 2026-08-12 03:57:47 on dist-test-slave-8tdl"
server_key: "9d1b2fcc0add5a5ff7e890cd4501d187"
server_key_iv: "47cf33f11e544cb3a89e036e99b889de"
server_key_version: "encryptionkey@0"
I20260812 03:57:47.179818 5378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 03:57:47.180763 5392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:47.181109 5378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:47.181241 5378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "5198357f2d5c4ae8b1bcf6f997c6b7ba"
format_stamp: "Formatted at 2026-08-12 03:57:47 on dist-test-slave-8tdl"
server_key: "9d1b2fcc0add5a5ff7e890cd4501d187"
server_key_iv: "47cf33f11e544cb3a89e036e99b889de"
server_key_version: "encryptionkey@0"
I20260812 03:57:47.181334 5378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:47.197922 5378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:47.198334 5378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:47.198522 5378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:47.203173 5378 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:40827
I20260812 03:57:47.203403 5451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:40827 every 8 connection(s)
I20260812 03:57:47.203682 5378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:47.204921 5452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:47.206864 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 5378
I20260812 03:57:47.206979 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:47.207175 31919 external_mini_cluster.cc:1428] Setting key b73105e620f77075ddc2bae76f2bfbad
I20260812 03:57:47.207935 5452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba: Bootstrap starting.
I20260812 03:57:47.208630 5452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:47.208976 5452 log.cc:824] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:47.209812 5452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba: No bootstrap required, opened a new log
I20260812 03:57:47.211787 5452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5198357f2d5c4ae8b1bcf6f997c6b7ba" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40827 } }
I20260812 03:57:47.212000 5452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:47.212041 5452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5198357f2d5c4ae8b1bcf6f997c6b7ba, State: Initialized, Role: FOLLOWER
I20260812 03:57:47.212191 5452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5198357f2d5c4ae8b1bcf6f997c6b7ba" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40827 } }
I20260812 03:57:47.212288 5452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:47.212313 5452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:47.212344 5452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:47.213212 5455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:47.207770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.31.43.254:40827)
0812 03:57:47.208074 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:47.208089 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:47.208118 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:57:47.208570 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:47.208578 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:47.208654 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:47.208861 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:47.209073 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.210159 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.211052 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.211468 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.211620 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:47.212424 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:47.212428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:47.212429 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:47.212648 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:47.212692 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:47.212707 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:47.212780 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:47.212856 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:57:47.212945 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260812 03:57:47.213200 5452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5198357f2d5c4ae8b1bcf6f997c6b7ba" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40827 } }
I20260812 03:57:47.213361 5452 leader_election.cc:302] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5198357f2d5c4ae8b1bcf6f997c6b7ba; no voters:
I20260812 03:57:47.213572 5452 leader_election.cc:288] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:47.213639 5457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:47.213805 5457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [term 1 LEADER]: Becoming Leader. State: Replica: 5198357f2d5c4ae8b1bcf6f997c6b7ba, State: Running, Role: LEADER
I20260812 03:57:47.213892 5457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5198357f2d5c4ae8b1bcf6f997c6b7ba" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40827 } }
I20260812 03:57:47.213907 5452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:47.214370 5459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5198357f2d5c4ae8b1bcf6f997c6b7ba. Latest consensus state: current_term: 1 leader_uuid: "5198357f2d5c4ae8b1bcf6f997c6b7ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5198357f2d5c4ae8b1bcf6f997c6b7ba" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40827 } } }
I20260812 03:57:47.214354 5460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5198357f2d5c4ae8b1bcf6f997c6b7ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5198357f2d5c4ae8b1bcf6f997c6b7ba" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40827 } } }
I20260812 03:57:47.214424 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:57:47.214455 5459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:47.214466 5460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:47.214709 5464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:47.215323 5464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:47.217219 5464 catalog_manager.cc:1347] Generated new cluster ID: 9efebfa83045416d9c460250fa1d333c
I20260812 03:57:47.217283 5464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:47.275820 5464 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:47.276584 5464 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:47.284924 5464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5198357f2d5c4ae8b1bcf6f997c6b7ba: Generated new TSK 0
I20260812 03:57:47.285203 5464 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 03:57:47.314170 5485 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 03:57:47.314361 5485 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 03:57:47.314563 5455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:47.298519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42094 (local address 127.31.43.254:40827)
0812 03:57:47.298579 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:47.298582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:47.298817 (+ 235us) server_negotiation.cc:407] Connection header received
0812 03:57:47.298941 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:47.298941 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:47.298954 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:47.299013 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:57:47.299211 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.299700 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.300870 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.301067 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.301135 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:47.301399 (+ 264us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:57:47.314278 (+ 12879us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34443/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 03:57:47.314482 (+ 204us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42094: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34443/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":28}
W20260812 03:57:47.314674 5455 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42094: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34443/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 03:57:47.315389 5483 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:47.298433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40827 (local address 127.0.0.1:42094)
0812 03:57:47.298722 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:47.298737 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:47.298828 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:47.299055 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:47.299056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:47.299058 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:57:47.299140 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:47.299143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.299787 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.299788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:47.300762 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:47.300766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.301179 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.301180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:47.301251 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:47.301280 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:57:47.314596 (+ 13316us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34443/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 03:57:47.314778 (+ 182us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.43.254:40827: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34443/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":116,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.31.43.254:40827: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34443/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 03:57:47.316047 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 5378
2026-08-12T03:57:47Z chronyd exiting
I20260812 03:57:47.357097 31919 test_util.cc:175] -----------------------------------------------
I20260812 03:57:47.357196 31919 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786507041847525-31919-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (300 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 03:57:47.358167 31919 mini_oidc.cc:152] Starting JWKS server
I20260812 03:57:47.358968 31919 webserver.cc:460] Webserver started at https://127.0.0.1:43521/ using document root <none> and password file <none>
I20260812 03:57:47.359022 31919 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:57:47.359198 31919 webserver.cc:460] Webserver started at http://127.0.0.1:37941/ using document root <none> and password file <none>
2026-08-12T03:57:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:47Z Disabled control of system clock
I20260812 03:57:47.370311 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:34927
--webserver_interface=127.31.43.254
--webserver_port=0
--builtin_ntp_servers=127.31.43.212:42065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:34927
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43521/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/testchainca.cert with env {}
W20260812 03:57:47.459717 5492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:47.459998 5492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:47.460039 5492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:47.461299 5492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:47.461376 5492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:47.461398 5492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:47.461421 5492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:47.461447 5492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:57:47.461480 5492 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:57:47.461508 5492 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43521/jwks.json
I20260812 03:57:47.463201 5492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:34927
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:34927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43521/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:47.463500 5492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:47.463738 5492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:47.469617 5498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:47.469704 5492 server_base.cc:1034] running on GCE node
W20260812 03:57:47.469617 5500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:47.469687 5497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:47.470417 5492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:47.470887 5492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:47.472222 5492 hybrid_clock.cc:648] HybridClock initialized: now 1786507067472196 us; error 43 us; skew 500 ppm
I20260812 03:57:47.473899 5492 webserver.cc:460] Webserver started at http://127.31.43.254:41327/ using document root <none> and password file <none>
I20260812 03:57:47.474267 5492 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:47.474354 5492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:47.474501 5492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:47.475840 5492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "0b3bb082d3114cb8be80e3362ee4e25e"
format_stamp: "Formatted at 2026-08-12 03:57:47 on dist-test-slave-8tdl"
server_key: "468af90b4080597e6063cbee53a278bc"
server_key_iv: "180c8750f4fad5b4df0f1b344d763724"
server_key_version: "encryptionkey@0"
I20260812 03:57:47.476297 5492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "0b3bb082d3114cb8be80e3362ee4e25e"
format_stamp: "Formatted at 2026-08-12 03:57:47 on dist-test-slave-8tdl"
server_key: "468af90b4080597e6063cbee53a278bc"
server_key_iv: "180c8750f4fad5b4df0f1b344d763724"
server_key_version: "encryptionkey@0"
I20260812 03:57:47.477941 5492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:47.478935 5507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:47.479252 5492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:47.479375 5492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "0b3bb082d3114cb8be80e3362ee4e25e"
format_stamp: "Formatted at 2026-08-12 03:57:47 on dist-test-slave-8tdl"
server_key: "468af90b4080597e6063cbee53a278bc"
server_key_iv: "180c8750f4fad5b4df0f1b344d763724"
server_key_version: "encryptionkey@0"
I20260812 03:57:47.479463 5492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:47.496450 5492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:47.496860 5492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:47.497045 5492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:47.501873 5492 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:34927
I20260812 03:57:47.501914 5566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:34927 every 8 connection(s)
I20260812 03:57:47.502349 5492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:47.503338 5568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:47.506358 5568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e: Bootstrap starting.
I20260812 03:57:47.506577 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 5492
I20260812 03:57:47.506739 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:47.506951 31919 external_mini_cluster.cc:1428] Setting key 6ca0d3216aaa73544a49e1c479885296
I20260812 03:57:47.507175 5568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:47.507580 5568 log.cc:824] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:47.508440 5568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e: No bootstrap required, opened a new log
I20260812 03:57:47.511008 5568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3bb082d3114cb8be80e3362ee4e25e" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34927 } }
I20260812 03:57:47.511165 5568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:47.511189 5568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b3bb082d3114cb8be80e3362ee4e25e, State: Initialized, Role: FOLLOWER
I20260812 03:57:47.511339 5568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3bb082d3114cb8be80e3362ee4e25e" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34927 } }
I20260812 03:57:47.511412 5568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:47.511435 5568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:47.511462 5568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:47.512281 5568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3bb082d3114cb8be80e3362ee4e25e" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34927 } }
I20260812 03:57:47.512399 5568 leader_election.cc:302] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b3bb082d3114cb8be80e3362ee4e25e; no voters:
I20260812 03:57:47.512588 5568 leader_election.cc:288] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:47.512712 5573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:47.512961 5568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:47.512902 5573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [term 1 LEADER]: Becoming Leader. State: Replica: 0b3bb082d3114cb8be80e3362ee4e25e, State: Running, Role: LEADER
I20260812 03:57:47.513072 5573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3bb082d3114cb8be80e3362ee4e25e" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34927 } }
I20260812 03:57:47.513461 5575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b3bb082d3114cb8be80e3362ee4e25e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3bb082d3114cb8be80e3362ee4e25e" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34927 } } }
I20260812 03:57:47.513433 5573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b3bb082d3114cb8be80e3362ee4e25e. Latest consensus state: current_term: 1 leader_uuid: "0b3bb082d3114cb8be80e3362ee4e25e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3bb082d3114cb8be80e3362ee4e25e" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34927 } } }
I20260812 03:57:47.513530 5575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:47.513537 5573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:47.514066 5583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:47.514451 5571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:47.507660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.31.43.254:34927)
0812 03:57:47.507910 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:47.507918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:47.507945 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:57:47.508641 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:47.508650 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:47.508733 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:47.508894 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:47.509176 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.510605 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.511733 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.512400 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.512535 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:47.513298 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:47.513303 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:47.513303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:47.513697 (+ 394us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:47.513742 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:47.513757 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:47.514098 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:47.514133 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 03:57:47.514260 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":108,"threads_started":1}
I20260812 03:57:47.514714 5583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:47.515614 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:57:47.516680 5583 catalog_manager.cc:1347] Generated new cluster ID: 61a112d965f141709a6aa0dc0773bda4
I20260812 03:57:47.516741 5583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:47.534886 5583 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:47.535646 5583 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:47.541231 5583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b3bb082d3114cb8be80e3362ee4e25e: Generated new TSK 0
I20260812 03:57:47.541488 5583 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 03:57:47.563298 5601 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 03:57:47.563401 5601 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 03:57:47.563808 5571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:47.548531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35150 (local address 127.31.43.254:34927)
0812 03:57:47.548591 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:47.548593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:47.548649 (+ 56us) server_negotiation.cc:407] Connection header received
0812 03:57:47.548737 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:47.548738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:47.548748 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:47.548814 (+ 66us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:57:47.549054 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.549601 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.550792 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.551054 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.551119 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:47.551323 (+ 204us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:57:47.563553 (+ 12230us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 03:57:47.563751 (+ 198us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35150: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":29}
W20260812 03:57:47.563886 5571 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35150: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 03:57:47.563972 5599 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:47.548406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:34927 (local address 127.0.0.1:35150)
0812 03:57:47.548580 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:47.548593 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:47.548668 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:47.548837 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:47.548838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:47.548840 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:57:47.548914 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:47.548916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.549695 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.549696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:47.550671 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:47.550709 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.551141 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.551142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:47.551209 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:47.551216 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:57:47.563689 (+ 12473us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 03:57:47.563859 (+ 170us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.43.254:34927: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
I20260812 03:57:47.564260 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 5492
2026-08-12T03:57:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (247 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 03:57:47.605846 31919 mini_oidc.cc:152] Starting JWKS server
I20260812 03:57:47.606544 31919 webserver.cc:460] Webserver started at https://127.0.0.1:33411/ using document root <none> and password file <none>
I20260812 03:57:47.606628 31919 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:57:47.606823 31919 webserver.cc:460] Webserver started at http://127.0.0.1:35437/ using document root <none> and password file <none>
2026-08-12T03:57:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:47Z Disabled control of system clock
I20260812 03:57:47.617269 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:34765
--webserver_interface=127.31.43.254
--webserver_port=0
--builtin_ntp_servers=127.31.43.212:42405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:34765
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33411/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 03:57:47.705087 5608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:47.705317 5608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:47.705341 5608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:47.706506 5608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:47.706547 5608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:47.706560 5608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:47.706573 5608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:47.706585 5608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:57:47.706597 5608 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:57:47.706609 5608 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33411/jwks.json
I20260812 03:57:47.708022 5608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:34765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:34765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33411/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:47.708256 5608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:47.708520 5608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:47.713856 5614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:47.713824 5617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:47.713816 5615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:47.714228 5608 server_base.cc:1034] running on GCE node
I20260812 03:57:47.714444 5608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:47.714785 5608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:47.715960 5608 hybrid_clock.cc:648] HybridClock initialized: now 1786507067715949 us; error 41 us; skew 500 ppm
I20260812 03:57:47.717386 5608 webserver.cc:460] Webserver started at http://127.31.43.254:43125/ using document root <none> and password file <none>
I20260812 03:57:47.717650 5608 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:47.717725 5608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:47.717881 5608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:47.719296 5608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "b4cf41da845042d0854cc5fbc972893b"
format_stamp: "Formatted at 2026-08-12 03:57:47 on dist-test-slave-8tdl"
server_key: "d2e81e4cbf28a7f50cd33ea4926a46a3"
server_key_iv: "80bd483e7d20c1e8192e7c379bd819be"
server_key_version: "encryptionkey@0"
I20260812 03:57:47.719705 5608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "b4cf41da845042d0854cc5fbc972893b"
format_stamp: "Formatted at 2026-08-12 03:57:47 on dist-test-slave-8tdl"
server_key: "d2e81e4cbf28a7f50cd33ea4926a46a3"
server_key_iv: "80bd483e7d20c1e8192e7c379bd819be"
server_key_version: "encryptionkey@0"
I20260812 03:57:47.721437 5608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:57:47.722731 5625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:47.722945 5608 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:47.723084 5608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "b4cf41da845042d0854cc5fbc972893b"
format_stamp: "Formatted at 2026-08-12 03:57:47 on dist-test-slave-8tdl"
server_key: "d2e81e4cbf28a7f50cd33ea4926a46a3"
server_key_iv: "80bd483e7d20c1e8192e7c379bd819be"
server_key_version: "encryptionkey@0"
I20260812 03:57:47.723170 5608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:47.739254 5608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:47.739651 5608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:47.739806 5608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:47.744545 5608 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:34765
I20260812 03:57:47.744583 5683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:34765 every 8 connection(s)
I20260812 03:57:47.744957 5608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:47.745759 5684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:47.749104 5684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b: Bootstrap starting.
I20260812 03:57:47.749792 5684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:47.750089 5684 log.cc:824] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:47.750983 5684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b: No bootstrap required, opened a new log
I20260812 03:57:47.752676 5684 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4cf41da845042d0854cc5fbc972893b" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34765 } }
I20260812 03:57:47.752836 5684 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:47.752861 5684 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4cf41da845042d0854cc5fbc972893b, State: Initialized, Role: FOLLOWER
I20260812 03:57:47.753043 5684 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4cf41da845042d0854cc5fbc972893b" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34765 } }
I20260812 03:57:47.753090 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 5608
I20260812 03:57:47.753130 5684 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:47.753176 5684 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:47.753201 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:47.753232 5684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:47.753404 31919 external_mini_cluster.cc:1428] Setting key f8c2346695028ddf26f9148eb8406c89
I20260812 03:57:47.753993 5684 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4cf41da845042d0854cc5fbc972893b" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34765 } }
I20260812 03:57:47.754125 5684 leader_election.cc:302] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4cf41da845042d0854cc5fbc972893b; no voters:
I20260812 03:57:47.754321 5684 leader_election.cc:288] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:47.754352 5690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:47.754601 5690 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [term 1 LEADER]: Becoming Leader. State: Replica: b4cf41da845042d0854cc5fbc972893b, State: Running, Role: LEADER
I20260812 03:57:47.754624 5684 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:47.754853 5690 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4cf41da845042d0854cc5fbc972893b" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34765 } }
I20260812 03:57:47.755437 5692 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4cf41da845042d0854cc5fbc972893b. Latest consensus state: current_term: 1 leader_uuid: "b4cf41da845042d0854cc5fbc972893b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4cf41da845042d0854cc5fbc972893b" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34765 } } }
I20260812 03:57:47.755551 5692 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:47.755787 5700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:47.755911 5691 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4cf41da845042d0854cc5fbc972893b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4cf41da845042d0854cc5fbc972893b" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34765 } } }
I20260812 03:57:47.756004 5691 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:47.756515 5700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:47.759884 5700 catalog_manager.cc:1347] Generated new cluster ID: 8f92c8daf68e450e9fef21cb0a857e5d
I20260812 03:57:47.759966 5700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:47.760715 5688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:47.753994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54212 (local address 127.31.43.254:34765)
0812 03:57:47.754275 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:47.754286 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:47.754331 (+ 45us) server_negotiation.cc:407] Connection header received
0812 03:57:47.754519 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:47.754528 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:47.754620 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:47.754901 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:47.755129 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.757180 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.758311 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.758931 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.759073 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:47.759879 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:47.759883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:47.759883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:47.760124 (+ 241us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:47.760172 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:47.760188 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:47.760258 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:47.760392 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 03:57:47.760487 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":92,"threads_started":1}
I20260812 03:57:47.761714 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:57:47.779636 5700 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:47.780436 5700 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:47.790762 5700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4cf41da845042d0854cc5fbc972893b: Generated new TSK 0
I20260812 03:57:47.791051 5700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:57:47.809978 5688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:47.806108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.31.43.254:34765)
0812 03:57:47.806189 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:47.806191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:47.806317 (+ 126us) server_negotiation.cc:407] Connection header received
0812 03:57:47.806479 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:47.806480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:47.806491 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:47.806581 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:57:47.806766 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.807317 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.808175 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.808412 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.808495 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:47.808710 (+ 215us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:57:47.809729 (+ 1019us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33411/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 03:57:47.809883 (+ 154us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54218: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33411/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 03:57:47.810050 5688 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54218: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33411/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 03:57:47.810070 5716 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:47.806021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:34765 (local address 127.0.0.1:54218)
0812 03:57:47.806246 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:47.806256 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:47.806338 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:47.806583 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:47.806584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:47.806587 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:57:47.806660 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:47.806662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.807431 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.807432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:47.808076 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:47.808078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:47.808498 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:47.808500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:47.808567 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:47.808580 (+ 13us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:57:47.809842 (+ 1262us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33411/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 03:57:47.809967 (+ 125us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.43.254:34765: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33411/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":127,"threads_started":1}
I20260812 03:57:47.810523 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 5608
E20260812 03:57:47.821318 5717 webserver.cc:533] Webserver: sslize failed:
E20260812 03:57:47.821435 5717 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T03:57:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (246 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T03:57:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:47Z Disabled control of system clock
I20260812 03:57:47.861723 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:34031
--webserver_interface=127.31.43.254
--webserver_port=0
--builtin_ntp_servers=127.31.43.212:43013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:34031
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/testchainca.cert with env {}
W20260812 03:57:47.948930 5722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:47.949190 5722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:47.949216 5722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:47.950541 5722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:47.950596 5722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:47.950609 5722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:47.950623 5722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:47.950635 5722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:57:47.950647 5722 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:57:47.950747 5722 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 03:57:47.952553 5722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:43013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:34031
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:34031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:47.952860 5722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:47.953118 5722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:47.958668 5727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:47.958730 5730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:47.958668 5728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:47.958871 5722 server_base.cc:1034] running on GCE node
I20260812 03:57:47.959252 5722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:47.959578 5722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:47.960759 5722 hybrid_clock.cc:648] HybridClock initialized: now 1786507067960747 us; error 53 us; skew 500 ppm
I20260812 03:57:47.962325 5722 webserver.cc:460] Webserver started at http://127.31.43.254:33439/ using document root <none> and password file <none>
I20260812 03:57:47.962594 5722 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:47.962670 5722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:47.962858 5722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:47.964421 5722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "24e1a5ab5c46445cb777163f539c5d51"
format_stamp: "Formatted at 2026-08-12 03:57:47 on dist-test-slave-8tdl"
server_key: "784364db5130a5f2b9983d0fd6635cd5"
server_key_iv: "9e8f76c7687c2901e3d0401b5766c204"
server_key_version: "encryptionkey@0"
I20260812 03:57:47.964850 5722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "24e1a5ab5c46445cb777163f539c5d51"
format_stamp: "Formatted at 2026-08-12 03:57:47 on dist-test-slave-8tdl"
server_key: "784364db5130a5f2b9983d0fd6635cd5"
server_key_iv: "9e8f76c7687c2901e3d0401b5766c204"
server_key_version: "encryptionkey@0"
I20260812 03:57:47.966491 5722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:47.967563 5736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:47.967821 5722 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:47.967927 5722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "24e1a5ab5c46445cb777163f539c5d51"
format_stamp: "Formatted at 2026-08-12 03:57:47 on dist-test-slave-8tdl"
server_key: "784364db5130a5f2b9983d0fd6635cd5"
server_key_iv: "9e8f76c7687c2901e3d0401b5766c204"
server_key_version: "encryptionkey@0"
I20260812 03:57:47.968096 5722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:47.995750 5722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:47.996157 5722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:47.996345 5722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:48.001271 5722 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:34031
I20260812 03:57:48.001344 5799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:34031 every 8 connection(s)
I20260812 03:57:48.001788 5722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:48.003067 5800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:48.005642 5800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51: Bootstrap starting.
I20260812 03:57:48.006255 5800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:48.006547 5800 log.cc:824] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:48.007490 5800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51: No bootstrap required, opened a new log
I20260812 03:57:48.007671 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 5722
I20260812 03:57:48.007807 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:48.008002 31919 external_mini_cluster.cc:1428] Setting key 52694ef17b1a8fd893b21725fc4976ff
I20260812 03:57:48.010263 5800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e1a5ab5c46445cb777163f539c5d51" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34031 } }
I20260812 03:57:48.010478 5800 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:48.010529 5800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24e1a5ab5c46445cb777163f539c5d51, State: Initialized, Role: FOLLOWER
I20260812 03:57:48.010701 5800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e1a5ab5c46445cb777163f539c5d51" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34031 } }
I20260812 03:57:48.010805 5800 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:48.010845 5800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:48.010900 5800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:48.011662 5800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e1a5ab5c46445cb777163f539c5d51" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34031 } }
I20260812 03:57:48.011802 5800 leader_election.cc:302] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24e1a5ab5c46445cb777163f539c5d51; no voters:
I20260812 03:57:48.012003 5800 leader_election.cc:288] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:48.012034 5806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:48.012279 5806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [term 1 LEADER]: Becoming Leader. State: Replica: 24e1a5ab5c46445cb777163f539c5d51, State: Running, Role: LEADER
I20260812 03:57:48.012310 5800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:48.012395 5806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e1a5ab5c46445cb777163f539c5d51" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34031 } }
I20260812 03:57:48.012799 5808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24e1a5ab5c46445cb777163f539c5d51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e1a5ab5c46445cb777163f539c5d51" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34031 } } }
I20260812 03:57:48.012791 5806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24e1a5ab5c46445cb777163f539c5d51. Latest consensus state: current_term: 1 leader_uuid: "24e1a5ab5c46445cb777163f539c5d51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e1a5ab5c46445cb777163f539c5d51" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34031 } } }
I20260812 03:57:48.012902 5808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:48.012909 5806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:48.013171 5814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:48.013773 5814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:48.015915 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:48.008696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54670 (local address 127.31.43.254:34031)
0812 03:57:48.008947 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:48.008957 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:48.008987 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:57:48.009588 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:48.009598 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:48.009699 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:48.009843 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:48.010218 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:48.011760 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:48.012967 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:48.013685 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:48.013796 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:48.015109 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:48.015114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:48.015114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:48.015317 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:48.015357 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:48.015370 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:48.015463 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:48.015580 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 03:57:48.015679 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":116,"threads_started":1}
I20260812 03:57:48.016360 5814 catalog_manager.cc:1347] Generated new cluster ID: f613762455bc4f8e962603a61d47cb97
I20260812 03:57:48.016422 5814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:48.017072 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:57:48.023635 5814 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:48.024246 5814 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:48.030052 5814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24e1a5ab5c46445cb777163f539c5d51: Generated new TSK 0
I20260812 03:57:48.030282 5814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:57:48.913882 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:48.041485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54682 (local address 127.31.43.254:34031)
0812 03:57:48.041562 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:48.041564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:48.041636 (+ 72us) server_negotiation.cc:407] Connection header received
0812 03:57:48.041705 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:48.041707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:48.041719 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:48.041807 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:57:48.042077 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:48.042888 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:48.043741 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:48.044062 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:48.044134 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:48.044344 (+ 210us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:57:48.913629 (+869285us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 03:57:48.913815 (+ 186us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54682: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 03:57:48.913956 5804 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54682: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 03:57:48.914191 5830 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:48.041246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:34031 (local address 127.0.0.1:54682)
0812 03:57:48.041564 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:48.041575 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:48.041636 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:48.041857 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:48.041858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:48.041860 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:57:48.041989 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:48.041991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:48.043013 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:48.043014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:48.043649 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:48.043652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:48.044155 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:48.044156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:48.044208 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:48.044214 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:57:48.913811 (+869597us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 03:57:48.914025 (+ 214us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.43.254:34031: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":165,"threads_started":1}
I20260812 03:57:48.914633 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 5722
2026-08-12T03:57:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1085 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (108 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (100 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:49 dist-test-slave-8tdl krb5kdc[5843](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:49 dist-test-slave-8tdl krb5kdc[5843](info): set up 2 sockets
Aug 12 03:57:49 dist-test-slave-8tdl krb5kdc[5843](info): commencing operation
krb5kdc: starting...
W20260812 03:57:50.305007 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:57:50 dist-test-slave-8tdl krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:57:50.455808 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:44847
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:44847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:50.546193 5857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:50.546429 5857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:50.546455 5857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:50.547827 5857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:50.547876 5857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:50.547891 5857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:50.547911 5857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:50.547923 5857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:50.549484 5857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:44847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:44847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:50.549713 5857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:50.549913 5857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:50.555303 5866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:50.555454 5863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:50.555303 5862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:50.555738 5857 server_base.cc:1034] running on GCE node
I20260812 03:57:50.556089 5857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:50.556442 5857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:50.557650 5857 hybrid_clock.cc:648] HybridClock initialized: now 1786507070557633 us; error 53 us; skew 500 ppm
Aug 12 03:57:50 dist-test-slave-8tdl krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507070, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:50.560714 5857 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:57:50.561305 5857 webserver.cc:460] Webserver started at http://127.31.43.254:42773/ using document root <none> and password file <none>
I20260812 03:57:50.561676 5857 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:50.561762 5857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:50.561933 5857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:50.563205 5857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "b6a7bc5660c14953bf59ab51cb5bb0e1"
format_stamp: "Formatted at 2026-08-12 03:57:50 on dist-test-slave-8tdl"
server_key: "af8bd21af0a087deeff1eb8a7a06ceaa"
server_key_iv: "cef201f4b30db61432366135237ec7de"
server_key_version: "encryptionkey@0"
I20260812 03:57:50.563603 5857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "b6a7bc5660c14953bf59ab51cb5bb0e1"
format_stamp: "Formatted at 2026-08-12 03:57:50 on dist-test-slave-8tdl"
server_key: "af8bd21af0a087deeff1eb8a7a06ceaa"
server_key_iv: "cef201f4b30db61432366135237ec7de"
server_key_version: "encryptionkey@0"
I20260812 03:57:50.565131 5857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:57:50.566193 5873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:50.566427 5857 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:50.566586 5857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "b6a7bc5660c14953bf59ab51cb5bb0e1"
format_stamp: "Formatted at 2026-08-12 03:57:50 on dist-test-slave-8tdl"
server_key: "af8bd21af0a087deeff1eb8a7a06ceaa"
server_key_iv: "cef201f4b30db61432366135237ec7de"
server_key_version: "encryptionkey@0"
I20260812 03:57:50.566675 5857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:50.581069 5857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:50.583875 5857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:50.584088 5857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:50.589043 5857 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:44847
I20260812 03:57:50.589136 5943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:44847 every 8 connection(s)
I20260812 03:57:50.589529 5857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:50.590534 5944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:50.592700 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 5857
I20260812 03:57:50.592825 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:50.593050 31919 external_mini_cluster.cc:1428] Setting key 85a1f830da8aadf4c5dbc1a0502ce480
I20260812 03:57:50.593127 5944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1: Bootstrap starting.
I20260812 03:57:50.593905 5944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:50.594229 5944 log.cc:824] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:50.595019 5944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1: No bootstrap required, opened a new log
I20260812 03:57:50.596923 5944 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a7bc5660c14953bf59ab51cb5bb0e1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 44847 } }
I20260812 03:57:50.597102 5944 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:50.597151 5944 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6a7bc5660c14953bf59ab51cb5bb0e1, State: Initialized, Role: FOLLOWER
I20260812 03:57:50.597330 5944 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a7bc5660c14953bf59ab51cb5bb0e1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 44847 } }
I20260812 03:57:50.597431 5944 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:50.597477 5944 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:50.597527 5944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:50.598237 5944 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a7bc5660c14953bf59ab51cb5bb0e1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 44847 } }
I20260812 03:57:50.598380 5944 leader_election.cc:302] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6a7bc5660c14953bf59ab51cb5bb0e1; no voters:
I20260812 03:57:50.598591 5944 leader_election.cc:288] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:50.598618 5950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:50.598830 5950 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [term 1 LEADER]: Becoming Leader. State: Replica: b6a7bc5660c14953bf59ab51cb5bb0e1, State: Running, Role: LEADER
I20260812 03:57:50.598929 5944 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:50.598940 5950 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a7bc5660c14953bf59ab51cb5bb0e1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 44847 } }
I20260812 03:57:50.599346 5950 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6a7bc5660c14953bf59ab51cb5bb0e1. Latest consensus state: current_term: 1 leader_uuid: "b6a7bc5660c14953bf59ab51cb5bb0e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a7bc5660c14953bf59ab51cb5bb0e1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 44847 } } }
I20260812 03:57:50.599359 5952 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6a7bc5660c14953bf59ab51cb5bb0e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a7bc5660c14953bf59ab51cb5bb0e1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 44847 } } }
I20260812 03:57:50.599488 5952 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:50.599435 5950 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:50.599720 5960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:50.600255 5960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:57:50 dist-test-slave-8tdl krb5kdc[5843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:50.602052 5960 catalog_manager.cc:1347] Generated new cluster ID: c6f56f5f4ca34c0c8c34e50eaa7bf718
I20260812 03:57:50.602111 5960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:50.606452 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:50.593703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44640 (local address 127.31.43.254:44847)
0812 03:57:50.593905 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:50.593912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:50.593966 (+ 54us) server_negotiation.cc:407] Connection header received
0812 03:57:50.594523 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:50.594531 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:50.594583 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:50.594789 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:50.595561 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:50.596472 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:50.597304 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:50.597837 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:50.597931 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:50.602891 (+ 4960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:50.602899 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:50.602900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:50.602912 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:50.604769 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:50.605181 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:50.605184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:50.605185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:50.605285 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:50.605522 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:50.605522 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:50.605523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:50.605770 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:50.605956 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:50.606072 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 03:57:50.606247 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260812 03:57:50.626606 5960 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:50.627297 5960 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:50.632988 5960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6a7bc5660c14953bf59ab51cb5bb0e1: Generated new TSK 0
I20260812 03:57:50.633240 5960 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 03:57:50.671015 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:44847
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:50.770303 5974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:50.770542 5974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:50.770574 5974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:50.771962 5974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:50.772022 5974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:50.772053 5974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:50.773717 5974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:44847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:50.774024 5974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:50.774314 5974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:50.780304 5984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:50.780304 5980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:50.780349 5981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:50.780678 5974 server_base.cc:1034] running on GCE node
I20260812 03:57:50.780939 5974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:50.781213 5974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:50.782440 5974 hybrid_clock.cc:648] HybridClock initialized: now 1786507070782404 us; error 59 us; skew 500 ppm
Aug 12 03:57:50 dist-test-slave-8tdl krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507070, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:50.785382 5974 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 03:57:50.785881 5974 webserver.cc:460] Webserver started at http://127.31.43.193:43133/ using document root <none> and password file <none>
I20260812 03:57:50.786132 5974 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:50.786208 5974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:50.786369 5974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:50.787642 5974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "5a3e5acd09814252bb78ad32bf441d54"
format_stamp: "Formatted at 2026-08-12 03:57:50 on dist-test-slave-8tdl"
server_key: "8f12d849374cbcc90632ab89e0806965"
server_key_iv: "c5241b20ab1af04245166b28a00d01c6"
server_key_version: "encryptionkey@0"
I20260812 03:57:50.788074 5974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "5a3e5acd09814252bb78ad32bf441d54"
format_stamp: "Formatted at 2026-08-12 03:57:50 on dist-test-slave-8tdl"
server_key: "8f12d849374cbcc90632ab89e0806965"
server_key_iv: "c5241b20ab1af04245166b28a00d01c6"
server_key_version: "encryptionkey@0"
I20260812 03:57:50.789762 5974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:57:50.790912 5992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:50.791111 5974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:50.791237 5974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "5a3e5acd09814252bb78ad32bf441d54"
format_stamp: "Formatted at 2026-08-12 03:57:50 on dist-test-slave-8tdl"
server_key: "8f12d849374cbcc90632ab89e0806965"
server_key_iv: "c5241b20ab1af04245166b28a00d01c6"
server_key_version: "encryptionkey@0"
I20260812 03:57:50.791327 5974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:50.804374 5974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:50.806550 5974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:50.806809 5974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:50.807116 5974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:50.807471 5974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:50.807526 5974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:50.807571 5974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:50.807600 5974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:50.813807 5974 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:41361
I20260812 03:57:50.813862 6125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:41361 every 8 connection(s)
I20260812 03:57:50.814208 5974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:57:50.819067 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 5974
I20260812 03:57:50.819181 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:57:50.819398 31919 external_mini_cluster.cc:1428] Setting key a538f2631d6696e32c1881a3caaa434f
Aug 12 03:57:50 dist-test-slave-8tdl krb5kdc[5843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507070, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:50.824867 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:50.815116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:37505 (local address 127.31.43.254:44847)
0812 03:57:50.815204 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:50.815207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:50.815387 (+ 180us) server_negotiation.cc:407] Connection header received
0812 03:57:50.816351 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:50.816353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:50.816366 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:50.816476 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:50.817235 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:50.817821 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:50.818956 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:50.819323 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:50.819412 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:50.821776 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:50.821778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:50.821778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:50.821788 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:50.823308 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:50.823733 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:50.823735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:50.823735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:50.823805 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:50.824158 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:50.824159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:50.824160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:50.824294 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:50.824377 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:50.824669 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 03:57:50.824768 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:57:50.825095 6128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:50.815026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:44847 (local address 127.31.43.193:37505)
0812 03:57:50.815210 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:50.815255 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:50.816241 (+ 986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:50.816598 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:50.816602 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:50.816876 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:50.817133 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:50.817138 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:50.817911 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:50.817911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:50.818864 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:50.818866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:50.819422 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:50.819423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:50.819592 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:50.820194 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:50.820205 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:50.821677 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:50.823388 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:50.823396 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:50.823398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:50.823626 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:50.823885 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:50.823886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:50.823886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:50.824000 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:50.824366 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:50.824375 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:50.824528 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 03:57:50.824646 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 03:57:50.824821 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260812 03:57:50.825728 6126 heartbeater.cc:347] Connected to a master server at 127.31.43.254:44847
I20260812 03:57:50.825855 6126 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:50.826086 6126 heartbeater.cc:511] Master 127.31.43.254:44847 requested a full tablet report, sending...
I20260812 03:57:50.826777 5898 ts_manager.cc:194] Registered new tserver with Master: 5a3e5acd09814252bb78ad32bf441d54 (127.31.43.193:41361)
I20260812 03:57:50.827793 5898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:37505
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 03:57:50.855870 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:44847
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:50.943763 6131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:50.944006 6131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:50.944033 6131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:50.945315 6131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:50.945370 6131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:50.945398 6131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:50.947239 6131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:44847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:50.947543 6131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:50.947844 6131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:57:50.953734 6131 server_base.cc:1034] running on GCE node
W20260812 03:57:50.953697 6138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:50.953697 6141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:50.953724 6139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:50.954308 6131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:50.954550 6131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:50.955718 6131 hybrid_clock.cc:648] HybridClock initialized: now 1786507070955706 us; error 32 us; skew 500 ppm
Aug 12 03:57:50 dist-test-slave-8tdl krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507070, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:50.958528 6131 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 03:57:50.959105 6131 webserver.cc:460] Webserver started at http://127.31.43.194:41083/ using document root <none> and password file <none>
I20260812 03:57:50.959371 6131 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:50.959424 6131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:50.959568 6131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:50.960705 6131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "ed261e2d3b744bf4bfda7ff2c80356f1"
format_stamp: "Formatted at 2026-08-12 03:57:50 on dist-test-slave-8tdl"
server_key: "fb29198cd5e0b96b9e47e087d896756f"
server_key_iv: "e8e8f9d4c6fc5589d187141a5d790c02"
server_key_version: "encryptionkey@0"
I20260812 03:57:50.961104 6131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "ed261e2d3b744bf4bfda7ff2c80356f1"
format_stamp: "Formatted at 2026-08-12 03:57:50 on dist-test-slave-8tdl"
server_key: "fb29198cd5e0b96b9e47e087d896756f"
server_key_iv: "e8e8f9d4c6fc5589d187141a5d790c02"
server_key_version: "encryptionkey@0"
I20260812 03:57:50.962589 6131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:57:50.963570 6152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:50.963778 6131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:50.963879 6131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "ed261e2d3b744bf4bfda7ff2c80356f1"
format_stamp: "Formatted at 2026-08-12 03:57:50 on dist-test-slave-8tdl"
server_key: "fb29198cd5e0b96b9e47e087d896756f"
server_key_iv: "e8e8f9d4c6fc5589d187141a5d790c02"
server_key_version: "encryptionkey@0"
I20260812 03:57:50.963964 6131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:50.976979 6131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:50.979359 6131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:50.979539 6131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:50.979800 6131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:50.980136 6131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:50.980175 6131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:50.980198 6131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:50.980247 6131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:50.986117 6131 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:39303
I20260812 03:57:50.986222 6280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:39303 every 8 connection(s)
I20260812 03:57:50.986584 6131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:57:50.993171 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 6131
I20260812 03:57:50.993310 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:57:50.993477 31919 external_mini_cluster.cc:1428] Setting key d10333a6ffca9341b46dcaadf2bc5f45
Aug 12 03:57:50 dist-test-slave-8tdl krb5kdc[5843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507070, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:50.997669 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:50.987468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:45317 (local address 127.31.43.254:44847)
0812 03:57:50.987559 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:50.987562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:50.987777 (+ 215us) server_negotiation.cc:407] Connection header received
0812 03:57:50.988818 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:50.988819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:50.988831 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:50.988892 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:50.989597 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:50.990109 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:50.991285 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:50.991479 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:50.991569 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:50.994113 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:50.994115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:50.994115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:50.994125 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:50.995820 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:50.996461 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:50.996462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:50.996463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:50.996544 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:50.996902 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:50.996904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:50.996904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:50.997055 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:50.997144 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:50.997468 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 03:57:50.997571 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:57:50.997892 6284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:50.987405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:44847 (local address 127.31.43.194:45317)
0812 03:57:50.987642 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:50.987683 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:50.988697 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:50.988978 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:50.988982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:50.989259 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:50.989480 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:50.989485 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:50.990251 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:50.990252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:50.991188 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:50.991191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:50.991576 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:50.991577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:50.991695 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:50.992216 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:50.992225 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:50.994001 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:50.995914 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:50.995925 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:50.995928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:50.996324 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:50.996633 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:50.996634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:50.996635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:50.996808 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:50.997134 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:50.997142 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:50.997306 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 03:57:50.997441 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 03:57:50.997628 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":120,"threads_started":1}
I20260812 03:57:50.998451 6281 heartbeater.cc:347] Connected to a master server at 127.31.43.254:44847
I20260812 03:57:50.998562 6281 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:50.998813 6281 heartbeater.cc:511] Master 127.31.43.254:44847 requested a full tablet report, sending...
I20260812 03:57:50.999310 5898 ts_manager.cc:194] Registered new tserver with Master: ed261e2d3b744bf4bfda7ff2c80356f1 (127.31.43.194:39303)
I20260812 03:57:50.999728 5898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:45317
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 03:57:51.027444 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:44847
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:51.115826 6288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:51.116106 6288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:51.116151 6288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:51.117421 6288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:51.117501 6288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:51.117548 6288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:51.119289 6288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:44847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:51.119603 6288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:51.119868 6288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:51.125625 6299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:51.125664 6293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:51.125660 6294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:51.125757 6288 server_base.cc:1034] running on GCE node
I20260812 03:57:51.126225 6288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:51.126531 6288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:51.127693 6288 hybrid_clock.cc:648] HybridClock initialized: now 1786507071127676 us; error 37 us; skew 500 ppm
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507071, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:51.130817 6288 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 03:57:51.131345 6288 webserver.cc:460] Webserver started at http://127.31.43.195:34803/ using document root <none> and password file <none>
I20260812 03:57:51.131603 6288 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:51.131675 6288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:51.131834 6288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:51.132987 6288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "eac9ff48a90641ce946401039f50c91e"
format_stamp: "Formatted at 2026-08-12 03:57:51 on dist-test-slave-8tdl"
server_key: "30fb065f3950d0eaf8b4ae08e6b1376c"
server_key_iv: "cdfc7908241b271cf5745612285c063a"
server_key_version: "encryptionkey@0"
I20260812 03:57:51.133432 6288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "eac9ff48a90641ce946401039f50c91e"
format_stamp: "Formatted at 2026-08-12 03:57:51 on dist-test-slave-8tdl"
server_key: "30fb065f3950d0eaf8b4ae08e6b1376c"
server_key_iv: "cdfc7908241b271cf5745612285c063a"
server_key_version: "encryptionkey@0"
I20260812 03:57:51.135123 6288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:51.136142 6309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:51.136363 6288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:51.136482 6288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "eac9ff48a90641ce946401039f50c91e"
format_stamp: "Formatted at 2026-08-12 03:57:51 on dist-test-slave-8tdl"
server_key: "30fb065f3950d0eaf8b4ae08e6b1376c"
server_key_iv: "cdfc7908241b271cf5745612285c063a"
server_key_version: "encryptionkey@0"
I20260812 03:57:51.136608 6288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:51.158200 6288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:51.160842 6288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:51.161041 6288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:51.161330 6288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:51.161674 6288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:51.161731 6288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:51.161777 6288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:51.161808 6288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:51.168831 6288 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:46393
I20260812 03:57:51.168887 6434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:46393 every 8 connection(s)
I20260812 03:57:51.169325 6288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:57:51.174803 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 6288
I20260812 03:57:51.174929 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:57:51.175137 31919 external_mini_cluster.cc:1428] Setting key 1ad12c75137afac0d29e8422cc9b1d46
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[5843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507071, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:57:51.180316 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:51.170234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:35293 (local address 127.31.43.254:44847)
0812 03:57:51.170361 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:51.170365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:51.170583 (+ 218us) server_negotiation.cc:407] Connection header received
0812 03:57:51.171735 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:51.171736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:51.171751 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:51.171804 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:51.172513 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:51.173047 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:51.174120 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:51.174349 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:51.174431 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:51.177204 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:51.177206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:51.177206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:51.177214 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:51.178740 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:51.179176 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:51.179178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:51.179178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:51.179236 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:51.179498 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:51.179500 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:51.179500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:51.179649 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:51.179718 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:51.180145 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 03:57:51.180247 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 03:57:51.180348 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:51.170132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:44847 (local address 127.31.43.195:35293)
0812 03:57:51.170386 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:51.170438 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:51.171601 (+ 1163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:51.171882 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:51.171886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:51.172188 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:51.172398 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:51.172402 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:51.173158 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:51.173159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:51.174016 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:51.174018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:51.174443 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:51.174444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:51.174611 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:51.175291 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:51.175303 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:51.177087 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:51.178826 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:51.178834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:51.178837 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:51.179105 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:51.179297 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:51.179298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:51.179298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:51.179409 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:51.179718 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:51.179723 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:51.179856 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 03:57:51.179944 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 03:57:51.180144 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":122,"threads_started":1}
I20260812 03:57:51.181027 6435 heartbeater.cc:347] Connected to a master server at 127.31.43.254:44847
I20260812 03:57:51.181162 6435 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:51.181385 6435 heartbeater.cc:511] Master 127.31.43.254:44847 requested a full tablet report, sending...
I20260812 03:57:51.181907 5898 ts_manager.cc:194] Registered new tserver with Master: eac9ff48a90641ce946401039f50c91e (127.31.43.195:46393)
I20260812 03:57:51.182337 5898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:35293
I20260812 03:57:51.187492 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/test-admin.keytab.
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:51.208853 31919 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 03:57:51.210965 31919 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/krb5cc' of type FILE
W20260812 03:57:51.211023 6445 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:51.209975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37802 (local address 127.31.43.193:41361)
0812 03:57:51.210180 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:51.210185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:51.210196 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:57:51.210226 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:51.210228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:51.210244 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:51.210316 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:51.210839 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37802: BlockingRecv error: recv got EOF from 127.0.0.1:37802 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:51.212352 31919 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:51.213372 31919 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[5843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.193@KRBTEST.COM
I20260812 03:57:51.223354 6445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:51.214170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37804 (local address 127.31.43.193:41361)
0812 03:57:51.214255 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:51.214257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:51.214273 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:57:51.214301 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:51.214302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:51.214317 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:51.214377 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:51.214936 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:51.215775 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:51.216829 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:51.217188 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:51.217251 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:51.220023 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:51.220036 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:51.220036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:51.220045 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:51.221679 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:51.222082 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:51.222085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:51.222086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:51.222167 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:51.222471 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:51.222471 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:51.222472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:51.222635 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:51.222814 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:51.223125 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 03:57:51.223204 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 03:57:51.225602 6445 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:51.224950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37816 (local address 127.31.43.193:41361)
0812 03:57:51.225028 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:51.225030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:51.225042 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:57:51.225136 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:51.225137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:51.225148 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:51.225206 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:51.225558 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37816: BlockingRecv error: recv got EOF from 127.0.0.1:37816 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 03:57:51.225793 31919 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:51.226982 31919 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:51.228045 31919 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[5843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.193@KRBTEST.COM
I20260812 03:57:51.235941 6445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:51.228669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37822 (local address 127.31.43.193:41361)
0812 03:57:51.228733 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:51.228734 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:51.228747 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:57:51.228830 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:51.228830 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:51.228839 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:51.228886 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:51.229239 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:51.229767 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:51.230558 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:51.230849 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:51.230910 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:51.233161 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:51.233163 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:51.233163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:51.233171 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:51.234508 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:51.235125 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:51.235126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:51.235126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:51.235187 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:51.235393 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:51.235394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:51.235394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:51.235493 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:51.235577 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:51.235735 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 03:57:51.235805 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 03:57:51.238159 6445 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:51.237425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37828 (local address 127.31.43.193:41361)
0812 03:57:51.237494 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:51.237496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:51.237506 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:51.237557 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:51.237558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:51.237567 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:51.237616 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:51.238023 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37828: BlockingRecv error: recv got EOF from 127.0.0.1:37828 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 03:57:51.238328 31919 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786507041847525-31919-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:51.239516 31919 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:51.240612 31919 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[5843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.193@KRBTEST.COM
I20260812 03:57:51.249264 6445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:51.241342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37842 (local address 127.31.43.193:41361)
0812 03:57:51.241400 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:51.241402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:51.241431 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:57:51.241461 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:51.241462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:51.241469 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:51.241509 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:51.241907 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:51.242416 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:51.243526 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:51.243748 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:51.243822 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:51.246274 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:51.246276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:51.246276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:51.246284 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:51.247732 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:51.248165 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:51.248166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:51.248166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:51.248230 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:51.248549 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:51.248550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:51.248550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:51.248678 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:51.248759 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:51.249054 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 03:57:51.249185 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:57:51.250226 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 5974
I20260812 03:57:51.258754 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 6131
I20260812 03:57:51.267055 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 6288
I20260812 03:57:51.276347 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 5857
2026-08-12T03:57:51Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2154 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[6462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[6462](info): set up 2 sockets
Aug 12 03:57:51 dist-test-slave-8tdl krb5kdc[6462](info): commencing operation
krb5kdc: starting...
W20260812 03:57:52.451548 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:57:52 dist-test-slave-8tdl krb5kdc[6462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.254.keytab.
Entry for principal oryx/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.254.keytab.
I20260812 03:57:52.598749 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:35427
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.254.keytab
--principal=oryx/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:35427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:52.695612 6477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:52.695895 6477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:52.695940 6477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:52.697292 6477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:52.697371 6477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:52.697397 6477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:52.697422 6477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:52.697453 6477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:52.699206 6477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:35427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.254.keytab
--principal=oryx/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:35427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:52.699542 6477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:52.699810 6477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:52.705577 6483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:52.705610 6486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:52.705564 6484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:52.705998 6477 server_base.cc:1034] running on GCE node
I20260812 03:57:52.706313 6477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:52.706650 6477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:52.707868 6477 hybrid_clock.cc:648] HybridClock initialized: now 1786507072707851 us; error 30 us; skew 500 ppm
Aug 12 03:57:52 dist-test-slave-8tdl krb5kdc[6462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507072, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:52.710721 6477 init.cc:377] Logged in from keytab as oryx/127.31.43.254@KRBTEST.COM (short username oryx)
I20260812 03:57:52.711223 6477 webserver.cc:460] Webserver started at http://127.31.43.254:39257/ using document root <none> and password file <none>
I20260812 03:57:52.711467 6477 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:52.711539 6477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:52.711700 6477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:52.712900 6477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "52dbb396108240fd9ce5b7b34010b505"
format_stamp: "Formatted at 2026-08-12 03:57:52 on dist-test-slave-8tdl"
server_key: "fd5f9b4b6c6829fd1a022854065f624c"
server_key_iv: "f59bc72552ff24c97c4669a1ef95c330"
server_key_version: "encryptionkey@0"
I20260812 03:57:52.713305 6477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "52dbb396108240fd9ce5b7b34010b505"
format_stamp: "Formatted at 2026-08-12 03:57:52 on dist-test-slave-8tdl"
server_key: "fd5f9b4b6c6829fd1a022854065f624c"
server_key_iv: "f59bc72552ff24c97c4669a1ef95c330"
server_key_version: "encryptionkey@0"
I20260812 03:57:52.714946 6477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:52.715929 6493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:52.716249 6477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:52.716370 6477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "52dbb396108240fd9ce5b7b34010b505"
format_stamp: "Formatted at 2026-08-12 03:57:52 on dist-test-slave-8tdl"
server_key: "fd5f9b4b6c6829fd1a022854065f624c"
server_key_iv: "f59bc72552ff24c97c4669a1ef95c330"
server_key_version: "encryptionkey@0"
I20260812 03:57:52.716455 6477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:52.723855 6477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:52.726553 6477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:52.726832 6477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:52.731489 6477 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:35427
I20260812 03:57:52.731941 6477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:52.731920 6548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:35427 every 8 connection(s)
I20260812 03:57:52.732856 6549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:52.735461 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 6477
I20260812 03:57:52.735601 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:52.735801 31919 external_mini_cluster.cc:1428] Setting key d775b161464203d73028027e2c754866
I20260812 03:57:52.735913 6549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505: Bootstrap starting.
I20260812 03:57:52.736775 6549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:52.737165 6549 log.cc:824] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:52.738063 6549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505: No bootstrap required, opened a new log
I20260812 03:57:52.740940 6549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } }
I20260812 03:57:52.741477 6549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:52.741539 6549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52dbb396108240fd9ce5b7b34010b505, State: Initialized, Role: FOLLOWER
I20260812 03:57:52.741688 6549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } }
I20260812 03:57:52.741825 6549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:52.741868 6549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:52.741918 6549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:52.742973 6549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } }
I20260812 03:57:52.743088 6549 leader_election.cc:302] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52dbb396108240fd9ce5b7b34010b505; no voters:
I20260812 03:57:52.743315 6549 leader_election.cc:288] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:52.743355 6555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:52.743474 6555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 1 LEADER]: Becoming Leader. State: Replica: 52dbb396108240fd9ce5b7b34010b505, State: Running, Role: LEADER
I20260812 03:57:52.743574 6555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } }
I20260812 03:57:52.743646 6549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:52.744081 6556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } }
I20260812 03:57:52.744243 6556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: This master's current role is: LEADER
Aug 12 03:57:52 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:52.745289 6557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52dbb396108240fd9ce5b7b34010b505. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } }
I20260812 03:57:52.745379 6557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: This master's current role is: LEADER
W20260812 03:57:52.745854 6572 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:57:52.745914 6572 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:57:52.749884 6564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:52.749935 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:52.736496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48598 (local address 127.31.43.254:35427)
0812 03:57:52.736715 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:52.736724 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:52.736765 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:57:52.737512 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:52.737520 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:52.737576 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:52.737794 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:52.738553 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:52.740203 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:52.741134 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:52.742060 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:52.742215 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:52.745840 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:52.745845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:52.745845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:52.745858 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:52.748038 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:52.748518 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:52.748522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:52.748523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:52.748651 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:52.748898 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:52.748900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:52.748900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:52.749211 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:52.749441 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:52.749519 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 03:57:52.749692 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":95,"threads_started":1}
I20260812 03:57:52.750147 6564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:52.752653 6564 catalog_manager.cc:1347] Generated new cluster ID: 9eab5de793e84a0fb81fbd2348d322b7
I20260812 03:57:52.752751 6564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:52.760437 6564 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:52.761381 6564 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:52.767584 6564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505: Generated new TSK 0
I20260812 03:57:52.767843 6564 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab.
Entry for principal oryx/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab.
I20260812 03:57:52.802811 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:35427
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab
--principal=oryx/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:52.892253 6579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:52.892486 6579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:52.892518 6579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:52.893747 6579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:52.893795 6579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:52.893821 6579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:52.895409 6579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab
--principal=oryx/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:35427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:52.895682 6579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:52.895941 6579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:52.901422 6586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:52.901414 6584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:52.901670 6579 server_base.cc:1034] running on GCE node
W20260812 03:57:52.901414 6588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:52.902055 6579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:52.902335 6579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:52.903501 6579 hybrid_clock.cc:648] HybridClock initialized: now 1786507072903487 us; error 36 us; skew 500 ppm
Aug 12 03:57:52 dist-test-slave-8tdl krb5kdc[6462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507072, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:52.906411 6579 init.cc:377] Logged in from keytab as oryx/127.31.43.193@KRBTEST.COM (short username oryx)
I20260812 03:57:52.906975 6579 webserver.cc:460] Webserver started at http://127.31.43.193:41963/ using document root <none> and password file <none>
I20260812 03:57:52.907231 6579 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:52.907313 6579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:52.907467 6579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:52.908586 6579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "bf97a7334d184009840330be17a07833"
format_stamp: "Formatted at 2026-08-12 03:57:52 on dist-test-slave-8tdl"
server_key: "61aada7451bf1a53dfc37ea1f792f6d7"
server_key_iv: "e3c70d43312d10f5fc54ac110d003056"
server_key_version: "encryptionkey@0"
I20260812 03:57:52.909006 6579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "bf97a7334d184009840330be17a07833"
format_stamp: "Formatted at 2026-08-12 03:57:52 on dist-test-slave-8tdl"
server_key: "61aada7451bf1a53dfc37ea1f792f6d7"
server_key_iv: "e3c70d43312d10f5fc54ac110d003056"
server_key_version: "encryptionkey@0"
I20260812 03:57:52.910566 6579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:52.911659 6600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:52.911885 6579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:52.911998 6579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "bf97a7334d184009840330be17a07833"
format_stamp: "Formatted at 2026-08-12 03:57:52 on dist-test-slave-8tdl"
server_key: "61aada7451bf1a53dfc37ea1f792f6d7"
server_key_iv: "e3c70d43312d10f5fc54ac110d003056"
server_key_version: "encryptionkey@0"
I20260812 03:57:52.912096 6579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:52.925168 6579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:52.927438 6579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:52.927644 6579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:52.927922 6579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:52.928256 6579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:52.928313 6579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:52.928357 6579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:52.928380 6579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:52.934803 6579 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:37083
I20260812 03:57:52.934969 6729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:37083 every 8 connection(s)
I20260812 03:57:52.935292 6579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:57:52.939448 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 6579
I20260812 03:57:52.939549 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:57:52.939782 31919 external_mini_cluster.cc:1428] Setting key 4b80f05e7b953079f5e9548bddb8dcfd
Aug 12 03:57:52 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507072, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:52.946539 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:52.936204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:50885 (local address 127.31.43.254:35427)
0812 03:57:52.936261 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:52.936263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:52.936444 (+ 181us) server_negotiation.cc:407] Connection header received
0812 03:57:52.937068 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:52.937070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:52.937083 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:52.937134 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:52.938198 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:52.938797 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:52.939998 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:52.940268 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:52.940352 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:52.943018 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:52.943020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:52.943021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:52.943031 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:52.944840 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:52.945320 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:52.945321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:52.945322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:52.945400 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:52.945719 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:52.945720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:52.945721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:52.945894 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:52.945969 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:52.946278 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 03:57:52.946388 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:57:52.946666 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:52.936075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.193:50885)
0812 03:57:52.936274 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:52.936325 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:52.936902 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:52.937215 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:52.937219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:52.937862 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:52.938079 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:52.938085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:52.938901 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:52.938902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:52.939880 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:52.939885 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:52.940362 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:52.940364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:52.940549 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:52.941136 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:52.941148 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:52.942900 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:52.944952 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:52.944960 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:52.944962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:52.945204 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:52.945489 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:52.945489 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:52.945490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:52.945643 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:52.945972 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:52.945980 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:52.946119 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 03:57:52.946223 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 03:57:52.946447 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":71,"threads_started":1}
I20260812 03:57:52.947423 6730 heartbeater.cc:347] Connected to a master server at 127.31.43.254:35427
I20260812 03:57:52.947561 6730 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:52.947805 6730 heartbeater.cc:511] Master 127.31.43.254:35427 requested a full tablet report, sending...
I20260812 03:57:52.948459 6512 ts_manager.cc:194] Registered new tserver with Master: bf97a7334d184009840330be17a07833 (127.31.43.193:37083)
I20260812 03:57:52.949128 6512 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:50885
WARNING: no policy specified for oryx/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab.
Entry for principal oryx/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab.
I20260812 03:57:52.974884 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:35427
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab
--principal=oryx/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:53.064536 6735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:53.064785 6735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:53.064813 6735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:53.066089 6735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:53.066138 6735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:53.066167 6735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:53.067775 6735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab
--principal=oryx/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:35427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:53.068055 6735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:53.068312 6735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:53.073917 6748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:53.073993 6741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:53.073987 6735 server_base.cc:1034] running on GCE node
W20260812 03:57:53.073917 6742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:53.074584 6735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:53.074946 6735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:53.076128 6735 hybrid_clock.cc:648] HybridClock initialized: now 1786507073076090 us; error 48 us; skew 500 ppm
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:53.079154 6735 init.cc:377] Logged in from keytab as oryx/127.31.43.194@KRBTEST.COM (short username oryx)
I20260812 03:57:53.079629 6735 webserver.cc:460] Webserver started at http://127.31.43.194:39825/ using document root <none> and password file <none>
I20260812 03:57:53.079881 6735 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:53.079954 6735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:53.080128 6735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:53.081373 6735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "325b8f3915404001925a596ef97ed930"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "741249e5327a4304b35aa4ecd437d294"
server_key_iv: "9f8346e032b1bdad1ed67579be1a4949"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.081780 6735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "325b8f3915404001925a596ef97ed930"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "741249e5327a4304b35aa4ecd437d294"
server_key_iv: "9f8346e032b1bdad1ed67579be1a4949"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.083765 6735 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 03:57:53.084702 6757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.084923 6735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:53.085095 6735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "325b8f3915404001925a596ef97ed930"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "741249e5327a4304b35aa4ecd437d294"
server_key_iv: "9f8346e032b1bdad1ed67579be1a4949"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.085180 6735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:53.100833 6735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:53.103495 6735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:53.103718 6735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:53.104003 6735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:53.104367 6735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:53.104431 6735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.104475 6735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:53.104503 6735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.110790 6735 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:33159
I20260812 03:57:53.110867 6884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:33159 every 8 connection(s)
I20260812 03:57:53.111235 6735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:57:53.111591 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 6735
I20260812 03:57:53.111738 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:57:53.111999 31919 external_mini_cluster.cc:1428] Setting key 5e3863cf1850692e99708ec6fe1df8be
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.31.43.195@KRBTEST.COM; defaulting to no policy
I20260812 03:57:53.122819 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.112291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:35919 (local address 127.31.43.254:35427)
0812 03:57:53.112357 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.112359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.112618 (+ 259us) server_negotiation.cc:407] Connection header received
0812 03:57:53.113215 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.113216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.113229 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.113283 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.114371 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.115078 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.116275 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.116609 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.116688 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.119512 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.119513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.119514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.119523 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.121156 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.121630 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.121631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.121632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.121703 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.121973 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.121975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.121975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.122133 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.122214 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.122524 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.122660 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:57:53.122943 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.112246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.194:35919)
0812 03:57:53.112462 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.112519 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.113094 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.113357 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.113362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.113974 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.114251 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.114257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.115194 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.115195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.116155 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.116158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.116710 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.116711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.116836 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.117558 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.117567 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.119395 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.121272 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.121282 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.121284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.121531 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.121778 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.121778 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.121779 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.121901 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.122217 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.122225 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.122379 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.122489 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.122672 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":104,"threads_started":1}
I20260812 03:57:53.123538 6885 heartbeater.cc:347] Connected to a master server at 127.31.43.254:35427
I20260812 03:57:53.123641 6885 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:53.123867 6885 heartbeater.cc:511] Master 127.31.43.254:35427 requested a full tablet report, sending...
I20260812 03:57:53.124359 6512 ts_manager.cc:194] Registered new tserver with Master: 325b8f3915404001925a596ef97ed930 (127.31.43.194:33159)
I20260812 03:57:53.124825 6512 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:35919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab.
Entry for principal oryx/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab.
I20260812 03:57:53.142838 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:35427
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab
--principal=oryx/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:53.234089 6890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:53.234344 6890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:53.234371 6890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:53.235844 6890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:53.235903 6890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:53.235934 6890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:53.237632 6890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab
--principal=oryx/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:35427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:53.237936 6890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:53.238173 6890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:53.243968 6897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:53.244000 6902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:53.244215 6898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:53.244510 6890 server_base.cc:1034] running on GCE node
I20260812 03:57:53.244767 6890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:53.245110 6890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:53.246322 6890 hybrid_clock.cc:648] HybridClock initialized: now 1786507073246320 us; error 93 us; skew 500 ppm
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:53.249243 6890 init.cc:377] Logged in from keytab as oryx/127.31.43.195@KRBTEST.COM (short username oryx)
I20260812 03:57:53.249747 6890 webserver.cc:460] Webserver started at http://127.31.43.195:40727/ using document root <none> and password file <none>
I20260812 03:57:53.250021 6890 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:53.250097 6890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:53.250259 6890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:53.251461 6890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "76634ab8eb914593903c21a6307a1a7e"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "5dd8589da9c103694dda3951cb0e8b53"
server_key_iv: "c05616577278b0e340df28a9fb8dcfd4"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.251957 6890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "76634ab8eb914593903c21a6307a1a7e"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "5dd8589da9c103694dda3951cb0e8b53"
server_key_iv: "c05616577278b0e340df28a9fb8dcfd4"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.253522 6890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:57:53.254403 6909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.254595 6890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:53.254716 6890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "76634ab8eb914593903c21a6307a1a7e"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "5dd8589da9c103694dda3951cb0e8b53"
server_key_iv: "c05616577278b0e340df28a9fb8dcfd4"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.254810 6890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:53.270380 6890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:53.272542 6890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:53.272729 6890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:53.273000 6890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:53.273355 6890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:53.273420 6890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.273468 6890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:53.273499 6890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.279975 6890 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:38991
I20260812 03:57:53.280017 7049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:38991 every 8 connection(s)
I20260812 03:57:53.280419 6890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:53.289830 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 6890
I20260812 03:57:53.289949 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:57:53.290138 31919 external_mini_cluster.cc:1428] Setting key 77f272b783eb294367f0137be124a179
I20260812 03:57:53.291296 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.281293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:44691 (local address 127.31.43.254:35427)
0812 03:57:53.281382 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.281385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.281628 (+ 243us) server_negotiation.cc:407] Connection header received
0812 03:57:53.282181 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.282182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.282193 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.282248 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.283418 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.283916 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.285191 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.285376 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.285432 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.287874 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.287875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.287876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.287883 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.289525 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.290084 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.290085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.290085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.290155 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.290461 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.290462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.290463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.290705 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.290799 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.291120 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.291213 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:57:53.291584 7052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.281249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.195:44691)
0812 03:57:53.281497 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.281539 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.282086 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.282309 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.282313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.283012 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.283322 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.283329 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.284169 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.284170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.285099 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.285101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.285481 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.285483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.285619 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.286142 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.286152 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.287774 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.289608 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.289615 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.289618 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.289881 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.290237 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.290238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.290238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.290359 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.290777 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.290785 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.290974 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.291096 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.291305 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":114,"threads_started":1}
I20260812 03:57:53.292140 7050 heartbeater.cc:347] Connected to a master server at 127.31.43.254:35427
I20260812 03:57:53.292259 7050 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:53.292455 7050 heartbeater.cc:511] Master 127.31.43.254:35427 requested a full tablet report, sending...
I20260812 03:57:53.292879 6512 ts_manager.cc:194] Registered new tserver with Master: 76634ab8eb914593903c21a6307a1a7e (127.31.43.195:38991)
I20260812 03:57:53.293376 6512 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:44691
I20260812 03:57:53.302515 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.43.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.253.keytab.
Entry for principal oryx/127.31.43.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.253.keytab.
I20260812 03:57:53.334996 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.253:32979
--webserver_interface=127.31.43.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.253.keytab
--principal=oryx/127.31.43.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:35427,127.31.43.253:32979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:53.422286 7056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:53.422554 7056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:53.422593 7056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:53.423897 7056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:53.423986 7056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:53.424015 7056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:53.424049 7056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:53.424074 7056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:53.425732 7056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:35427,127.31.43.253:32979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.253.keytab
--principal=oryx/127.31.43.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.253:32979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.43.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
W20260812 03:57:53.425899 7056 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.31.43.254:35427,127.31.43.253:32979'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 03:57:53.426074 7056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:53.426321 7056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:53.431766 7063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:53.431828 7066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:53.431922 7064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:53.432219 7056 server_base.cc:1034] running on GCE node
I20260812 03:57:53.432531 7056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:53.432863 7056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:53.434027 7056 hybrid_clock.cc:648] HybridClock initialized: now 1786507073434012 us; error 28 us; skew 500 ppm
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:53.437043 7056 init.cc:377] Logged in from keytab as oryx/127.31.43.253@KRBTEST.COM (short username oryx)
I20260812 03:57:53.437601 7056 webserver.cc:460] Webserver started at http://127.31.43.253:43695/ using document root <none> and password file <none>
I20260812 03:57:53.437842 7056 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:53.437928 7056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:53.438091 7056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:53.439335 7056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data/instance:
uuid: "32f35d95da9e45b78a40bacdc04b6c53"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "7752d359869a74ca3dd439caea34f9f5"
server_key_iv: "af277e97c63f157d968c199e6bf228a5"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.439780 7056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/wal/instance:
uuid: "32f35d95da9e45b78a40bacdc04b6c53"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "7752d359869a74ca3dd439caea34f9f5"
server_key_iv: "af277e97c63f157d968c199e6bf228a5"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.441282 7056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:57:53.442142 7074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.442340 7056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:53.442487 7056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/wal
uuid: "32f35d95da9e45b78a40bacdc04b6c53"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "7752d359869a74ca3dd439caea34f9f5"
server_key_iv: "af277e97c63f157d968c199e6bf228a5"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.442579 7056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:53.472283 7056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:53.475363 7056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:53.475612 7056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:53.480962 7056 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.253:32979
I20260812 03:57:53.481048 7136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.253:32979 every 8 connection(s)
I20260812 03:57:53.481505 7056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
I20260812 03:57:53.482049 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 7056
I20260812 03:57:53.482836 7137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:53.485425 7137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.43.253@KRBTEST.COM
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.253@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:53.494448 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.482822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46880 (local address 127.31.43.253:32979)
0812 03:57:53.483099 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.483117 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.483146 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:57:53.483767 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.483776 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.483835 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.483972 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.484837 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.485981 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.486885 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.487511 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.487600 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.490610 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.490616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.490617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.490623 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.492513 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.492991 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.492993 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.492994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.493073 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.493312 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.493313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.493313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.493556 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.493870 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.494038 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.494185 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":187,"threads_started":1}
I20260812 03:57:53.495028 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.486171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.31.43.254:35427)
0812 03:57:53.486242 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.486245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.486414 (+ 169us) server_negotiation.cc:407] Connection header received
0812 03:57:53.486496 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.486497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.486511 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.486591 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.487258 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.487866 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.488957 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.489261 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.489333 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.491870 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.491872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.491872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.491881 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.493532 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.493934 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.493936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.493936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.494000 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.494240 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.494241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.494242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.494453 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.494547 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.494853 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.494954 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:57:53.495124 7141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.486093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:48604)
0812 03:57:53.486298 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.486344 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.486436 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.486617 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.486620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.487074 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.487158 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.487160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.488013 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.488014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.488831 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.488833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.489343 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.489344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.489417 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.489902 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.489910 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.491762 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.493622 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.493629 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.493632 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.493847 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.494083 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.494083 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.494084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.494155 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.494574 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.494582 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.494649 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.494811 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.495017 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":139,"threads_started":1}
I20260812 03:57:53.495723 7137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.253@KRBTEST.COM for oryx/127.31.43.253@KRBTEST.COM
I20260812 03:57:53.502130 7141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.496107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:32979 (local address 127.0.0.1:46888)
0812 03:57:53.496152 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.496161 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.496200 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.496324 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.496324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.496468 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.496499 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.496501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.497134 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.497135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.497617 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.497618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.497969 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.497970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.498031 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.498420 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.498426 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.499672 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.501147 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.501148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.501149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.501308 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.501504 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.501505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.501505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.501558 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.501814 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.501815 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.501851 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.501920 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.502028 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 03:57:53.502220 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.496154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46888 (local address 127.31.43.253:32979)
0812 03:57:53.496188 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.496190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.496199 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:53.496256 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.496257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.496266 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.496320 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.496587 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.497054 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.497698 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.497895 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.497971 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.499763 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.499764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.499765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.499772 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.501043 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.501381 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.501382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.501382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.501438 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.501636 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.501637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.501637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.501748 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.501824 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.501950 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.502077 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":18}
I20260812 03:57:53.503913 7137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: Bootstrap starting.
I20260812 03:57:53.505024 7137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:53.505355 7137 log.cc:824] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:53.506253 7137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: No bootstrap required, opened a new log
I20260812 03:57:53.507767 7137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } }
I20260812 03:57:53.507905 7137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:53.507929 7137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32f35d95da9e45b78a40bacdc04b6c53, State: Initialized, Role: FOLLOWER
I20260812 03:57:53.508057 7137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } }
I20260812 03:57:53.508445 7143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } } }
I20260812 03:57:53.508534 7137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:53.508585 7143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:57:53.510416 7156 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:57:53.510484 7156 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:57:53.512647 7056 master_runner.cc:428] Detected that this master 32f35d95da9e45b78a40bacdc04b6c53 is joining an existing cluster
I20260812 03:57:53.512696 7056 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.43.253:32979
I20260812 03:57:53.522064 7163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.513892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:48608)
0812 03:57:53.514145 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.514160 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.514260 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.514502 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.514503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.514765 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.514884 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.514886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.516072 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.516073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.517030 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.517032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.517559 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.517560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.517633 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.518188 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.518193 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.518640 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.520751 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.520753 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.520753 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.521120 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.521351 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.521351 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.521352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.521421 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.521687 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.521688 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.521734 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.521828 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.521940 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":97,"threads_started":1}
I20260812 03:57:53.522135 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.513677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48608 (local address 127.31.43.254:35427)
0812 03:57:53.513750 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.513752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.514226 (+ 474us) server_negotiation.cc:407] Connection header received
0812 03:57:53.514386 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.514388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.514403 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.514470 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.515116 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.515923 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.517125 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.517433 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.517512 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.518766 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.518767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.518768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.518776 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.520663 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.521213 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.521214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.521215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.521280 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.521488 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.521488 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.521488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.521608 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.521695 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.521858 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.521940 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:57:53.522915 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.514230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46894 (local address 127.31.43.253:32979)
0812 03:57:53.514272 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.514274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.514936 (+ 662us) server_negotiation.cc:407] Connection header received
0812 03:57:53.514987 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.514988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.515015 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.515115 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.516466 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.517127 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.518048 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.518233 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.518311 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.519390 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.519392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.519392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.519399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.521272 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.521815 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.521816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.521816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.521863 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.522031 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.522032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.522032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.522145 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.522188 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.522739 (+ 551us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.522818 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:57:53.523432 7164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.514126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:32979 (local address 127.0.0.1:46894)
0812 03:57:53.514839 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.514850 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.514928 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.516062 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.516063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.516307 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.516379 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.516380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.517222 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.517223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.517939 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.517941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.518316 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.518317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.518384 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.518835 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.518840 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.519291 (+ 451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.521500 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.521501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.521501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.521729 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.521926 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.521927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.521927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.521972 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.522562 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.522563 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.522604 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.523224 (+ 620us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.523328 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":74,"threads_started":1}
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507072, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.254@KRBTEST.COM for oryx/127.31.43.253@KRBTEST.COM
I20260812 03:57:53.532486 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.525059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:32979 (local address 127.0.0.1:46902)
0812 03:57:53.525199 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.525241 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.525335 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.525467 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.525470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.525774 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.525830 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.525832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.526470 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.526471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.527158 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.527160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.527535 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.527536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.527624 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.528088 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.528092 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.529479 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.531212 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.531222 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.531225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.531454 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.531691 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.531692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.531692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.531763 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.532091 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.532100 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.532158 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.532257 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.532389 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260812 03:57:53.532474 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.525098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46902 (local address 127.31.43.253:32979)
0812 03:57:53.525135 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.525137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.525309 (+ 172us) server_negotiation.cc:407] Connection header received
0812 03:57:53.525403 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.525403 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.525413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.525463 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.525916 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.526378 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.527255 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.527443 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.527531 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.529579 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.529580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.529581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.529589 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.531118 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.531558 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.531560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.531560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.531620 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.531831 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.531832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.531872 (+ 40us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.531981 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.532085 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.532286 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.532374 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 03:57:53.533581 6512 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.43.253:32979: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53"
member_type: NON_VOTER
last_known_addr {
host: "127.31.43.253"
port: 32979
}
attrs {
promote: true
}
}
dest_uuid: "52dbb396108240fd9ce5b7b34010b505"
cas_config_opid_index: -1
I20260812 03:57:53.534080 6512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true } }
I20260812 03:57:53.534639 7166 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 32f35d95da9e45b78a40bacdc04b6c53 (127.31.43.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true } } }
I20260812 03:57:53.535046 7109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:53.535895 7166 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.43.253:32979
I20260812 03:57:53.536059 7109 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 FOLLOWER]: Refusing update from remote peer 52dbb396108240fd9ce5b7b34010b505: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 03:57:53.536042 7166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true } } }
I20260812 03:57:53.536109 7166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:53.536361 7056 master.cc:537] Master@127.31.43.253:32979 shutting down...
I20260812 03:57:53.536403 7166 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:57:53.537557 6498 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 03:57:53.538303 6498 consensus_peers.cc:487] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 -> Peer 32f35d95da9e45b78a40bacdc04b6c53 (127.31.43.253:32979): Couldn't send request to peer 32f35d95da9e45b78a40bacdc04b6c53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:57:53.539609 7056 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:53.539744 7056 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:53.539801 7056 tablet_replica.cc:330] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: stopping tablet replica
I20260812 03:57:53.553119 7056 master.cc:552] Master@127.31.43.253:32979 shutdown complete.
I20260812 03:57:53.553200 7056 master_runner.cc:305] Clearing existing system tablet
I20260812 03:57:53.553339 7056 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:53.553402 7056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:53.554126 7056 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260812 03:57:53.554746 7173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.554885 7056 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:53.554948 7056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:57:53.554973 7056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/wal
uuid: "32f35d95da9e45b78a40bacdc04b6c53"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "7752d359869a74ca3dd439caea34f9f5"
server_key_iv: "af277e97c63f157d968c199e6bf228a5"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.556046 7056 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:53.558043 7056 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 03:57:53.558130 7056 log.cc:1196] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 03:57:53.558359 7056 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: Deleting consensus metadata
I20260812 03:57:53.558499 7056 master_runner.cc:315] Copying system tablet from 127.31.43.254:35427
I20260812 03:57:53.559607 7056 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: tablet copy: Beginning tablet copy session from remote peer at address 127.31.43.254:35427
I20260812 03:57:53.566833 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.560082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48614 (local address 127.31.43.254:35427)
0812 03:57:53.560136 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.560138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.560281 (+ 143us) server_negotiation.cc:407] Connection header received
0812 03:57:53.560310 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.560310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.560320 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.560375 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.560808 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.561368 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.562468 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.562820 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.562905 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.564086 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.564087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.564087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.564094 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.565562 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.566018 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.566019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.566019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.566076 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.566283 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.566283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.566283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.566393 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.566446 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.566622 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.566754 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:57:53.566851 7181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.559977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:48614)
0812 03:57:53.560132 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.560145 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.560226 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.560418 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.560419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.560645 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.560730 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.560732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.561464 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.561465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.562355 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.562358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.562943 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.562944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.563019 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.563502 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.563507 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.563981 (+ 474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.565648 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.565649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.565650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.565894 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.566149 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.566150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.566151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.566219 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.566460 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.566461 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.566505 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.566590 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.566727 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260812 03:57:53.567706 6532 tablet_copy_service.cc:140] P 52dbb396108240fd9ce5b7b34010b505: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 32f35d95da9e45b78a40bacdc04b6c53 ({username='oryx', principal='oryx/127.31.43.253@KRBTEST.COM'} at 127.0.0.1:48614)
I20260812 03:57:53.567888 6532 tablet_copy_service.cc:161] P 52dbb396108240fd9ce5b7b34010b505: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 32f35d95da9e45b78a40bacdc04b6c53 at {username='oryx', principal='oryx/127.31.43.253@KRBTEST.COM'} at 127.0.0.1:48614: session id = 32f35d95da9e45b78a40bacdc04b6c53-00000000000000000000000000000000
I20260812 03:57:53.568907 6532 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:57:53.569634 7056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 03:57:53.570840 7056 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: tablet copy: Starting download of 0 data blocks...
I20260812 03:57:53.571022 7056 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: tablet copy: Starting download of 1 WAL segments...
I20260812 03:57:53.572100 7056 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:57:53.572970 6532 tablet_copy_service.cc:342] P 52dbb396108240fd9ce5b7b34010b505: Request end of tablet copy session 32f35d95da9e45b78a40bacdc04b6c53-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.43.253@KRBTEST.COM'} at 127.0.0.1:48614
I20260812 03:57:53.573036 6532 tablet_copy_service.cc:433] P 52dbb396108240fd9ce5b7b34010b505: ending tablet copy session 32f35d95da9e45b78a40bacdc04b6c53-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 32f35d95da9e45b78a40bacdc04b6c53
I20260812 03:57:53.575907 7056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:53.577224 7182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:53.577327 7183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:53.577337 7185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:53.577749 7056 server_base.cc:1034] running on GCE node
I20260812 03:57:53.577867 7056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:53.578014 7056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:53.579137 7056 hybrid_clock.cc:648] HybridClock initialized: now 1786507073579136 us; error 37 us; skew 500 ppm
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:53.580549 7056 init.cc:377] Logged in from keytab as oryx/127.31.43.253@KRBTEST.COM (short username oryx)
I20260812 03:57:53.580837 7056 webserver.cc:460] Webserver started at http://127.31.43.253:40469/ using document root <none> and password file <none>
I20260812 03:57:53.580976 7056 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:53.581024 7056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:53.581735 7056 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.582293 7192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.582473 7056 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.582542 7056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/wal
uuid: "32f35d95da9e45b78a40bacdc04b6c53"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "7752d359869a74ca3dd439caea34f9f5"
server_key_iv: "af277e97c63f157d968c199e6bf228a5"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.582607 7056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:53.587739 7056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:53.589564 7056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:53.594898 7056 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.253:32979
I20260812 03:57:53.595376 7253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.253:32979 every 8 connection(s)
I20260812 03:57:53.595400 7056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
I20260812 03:57:53.596688 7254 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:57:53.597209 7254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: Bootstrap starting.
I20260812 03:57:53.600697 7254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:57:53.600935 7254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: Bootstrap complete.
I20260812 03:57:53.601227 7254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true } }
I20260812 03:57:53.601325 7254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 32f35d95da9e45b78a40bacdc04b6c53, State: Initialized, Role: LEARNER
I20260812 03:57:53.601368 7254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true } }
I20260812 03:57:53.601593 7254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:53.601554 7258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true } } }
I20260812 03:57:53.601763 7258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:57:53.603935 7271 catalog_manager.cc:1259] Loaded cluster ID: 9eab5de793e84a0fb81fbd2348d322b7
I20260812 03:57:53.604141 7271 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: loading cluster ID for follower catalog manager: success
I20260812 03:57:53.604892 7271 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: acquiring CA information for follower catalog manager: success
I20260812 03:57:53.605203 7271 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:57:53.606051 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.596403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46912 (local address 127.31.43.253:32979)
0812 03:57:53.596559 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.596561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.596613 (+ 52us) server_negotiation.cc:407] Connection header received
0812 03:57:53.596695 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.596696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.596708 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.596779 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.597548 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.598601 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.599747 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.600069 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.600144 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.601796 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.601798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.601799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.601806 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.603918 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.605215 (+ 1297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.605216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.605217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.605289 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.605554 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.605555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.605555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.605670 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.605757 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.605915 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.605991 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for oryx/127.31.43.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.252.keytab.
Entry for principal oryx/127.31.43.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.252.keytab.
I20260812 03:57:53.645929 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.252:34789
--webserver_interface=127.31.43.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.252.keytab
--principal=oryx/127.31.43.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:35427,127.31.43.253:32979,127.31.43.252:34789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:53.754545 7274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:53.754854 7274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:53.754940 7274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:53.756228 7274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:53.756287 7274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:53.756302 7274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:53.756315 7274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:53.756327 7274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:53.758117 7274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:35427,127.31.43.253:32979,127.31.43.252:34789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.252.keytab
--principal=oryx/127.31.43.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.252:34789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.43.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:53.758400 7274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:53.758617 7274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:53.763944 7282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:53.764081 7285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:53.764118 7281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:53.764815 7274 server_base.cc:1034] running on GCE node
I20260812 03:57:53.765157 7274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:53.765604 7274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:53.766741 7274 hybrid_clock.cc:648] HybridClock initialized: now 1786507073766706 us; error 43 us; skew 500 ppm
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:53.770957 7274 init.cc:377] Logged in from keytab as oryx/127.31.43.252@KRBTEST.COM (short username oryx)
I20260812 03:57:53.771708 7274 webserver.cc:460] Webserver started at http://127.31.43.252:38027/ using document root <none> and password file <none>
I20260812 03:57:53.772042 7274 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:53.772146 7274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:53.772307 7274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:53.773964 7274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data/instance:
uuid: "9a0da90f06dc41838c0eade213130e3a"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "7b67f70669e2c1f8a5524c116fc10ded"
server_key_iv: "5035497fcf71102421900482dc0f33bf"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.774497 7274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/wal/instance:
uuid: "9a0da90f06dc41838c0eade213130e3a"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "7b67f70669e2c1f8a5524c116fc10ded"
server_key_iv: "5035497fcf71102421900482dc0f33bf"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.776721 7274 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260812 03:57:53.777750 7294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.777973 7274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:53.778090 7274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/wal
uuid: "9a0da90f06dc41838c0eade213130e3a"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "7b67f70669e2c1f8a5524c116fc10ded"
server_key_iv: "5035497fcf71102421900482dc0f33bf"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.778182 7274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:53.807569 7274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:53.810416 7274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:53.810590 7274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:53.817080 7274 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.252:34789
I20260812 03:57:53.817224 7352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.252:34789 every 8 connection(s)
I20260812 03:57:53.817663 7274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
I20260812 03:57:53.819239 7354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:53.820370 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 7274
I20260812 03:57:53.821590 7354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.252@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.43.252@KRBTEST.COM
I20260812 03:57:53.831984 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.822829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48620 (local address 127.31.43.254:35427)
0812 03:57:53.822875 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.822877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.823075 (+ 198us) server_negotiation.cc:407] Connection header received
0812 03:57:53.823154 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.823155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.823168 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.823220 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.823739 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.824307 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.825251 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.825451 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.825539 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.828170 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.828172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.828172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.828204 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.830194 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.830793 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.830794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.830795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.830873 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.831203 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.831205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.831205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.831359 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.831438 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.831788 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.831885 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:57:53.832199 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.822644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:48620)
0812 03:57:53.822943 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.822985 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.823095 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.823236 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.823239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.823536 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.823639 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.823643 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.824415 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.824416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.825153 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.825156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.825595 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.825596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.825689 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.826237 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.826246 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.828058 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.830291 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.830299 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.830302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.830646 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.830966 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.830966 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.830967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.831099 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.831441 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.831450 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.831631 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.831742 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.831928 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":22,"thread_start_us":144,"threads_started":1}
I20260812 03:57:53.832875 7354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:53.832993 7356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.821144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35164 (local address 127.31.43.252:34789)
0812 03:57:53.821369 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.821382 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.821412 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:57:53.821997 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.822005 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.822114 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.822300 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.823145 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.824109 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.824845 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.825293 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.825418 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.828786 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.828796 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.828796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.828804 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.831091 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.831521 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.831525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.831526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.831587 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.831838 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.831839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.831839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.831992 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.832696 (+ 704us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.832750 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.832858 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":169,"thread_start_us":101,"threads_started":1}
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.252@KRBTEST.COM for oryx/127.31.43.253@KRBTEST.COM
I20260812 03:57:53.841998 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.833265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:32979 (local address 127.0.0.1:46926)
0812 03:57:53.833370 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.833382 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.833421 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.833610 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.833611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.833778 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.833836 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.833838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.834817 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.834819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.835437 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.835440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.835853 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.835854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.835932 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.836433 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.836438 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.837884 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.840775 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.840777 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.840777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.840971 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.841255 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.841256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.841261 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.841319 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.841636 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.841637 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.841689 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.841765 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.841891 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 03:57:53.841997 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.833408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46926 (local address 127.31.43.253:32979)
0812 03:57:53.833481 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.833483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.833497 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:57:53.833522 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.833523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.833534 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.833606 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.833925 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.834670 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.835521 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.835769 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.835857 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.837988 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.837990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.837990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.837999 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.840661 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.841067 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.841069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.841069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.841168 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.841391 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.841392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.841393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.841531 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.841614 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.841806 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.841897 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:57:53.842782 7354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.252@KRBTEST.COM for oryx/127.31.43.252@KRBTEST.COM
I20260812 03:57:53.850382 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.843124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34789 (local address 127.0.0.1:35168)
0812 03:57:53.843178 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.843190 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.843232 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.843354 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.843356 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.843546 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.843597 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.843599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.844295 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.844296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.844914 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.844916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.845320 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.845321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.845370 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.845806 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.845811 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.847277 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.849281 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.849283 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.849283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.849458 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.849725 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.849726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.849726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.849784 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.850072 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.850074 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.850119 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.850201 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.850306 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 03:57:53.850505 7356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.843193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35168 (local address 127.31.43.252:34789)
0812 03:57:53.843232 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.843234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.843242 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:57:53.843284 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.843285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.843294 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.843345 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.843677 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.844178 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.845011 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.845234 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.845321 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.847396 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.847397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.847398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.847406 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.849186 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.849554 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.849556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.849556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.849647 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.849849 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.849849 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.849850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.849987 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.850044 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.850239 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.850357 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":21}
I20260812 03:57:53.852173 7354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: Bootstrap starting.
I20260812 03:57:53.853091 7354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:53.853401 7354 log.cc:824] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:53.854344 7354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: No bootstrap required, opened a new log
I20260812 03:57:53.856097 7354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } }
I20260812 03:57:53.856290 7354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:53.856346 7354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0da90f06dc41838c0eade213130e3a, State: Initialized, Role: FOLLOWER
I20260812 03:57:53.856495 7354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } }
I20260812 03:57:53.856942 7354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:53.857074 7359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } } }
I20260812 03:57:53.857174 7359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:57:53.859866 7371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:57:53.859972 7371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:57:53.863157 7274 master_runner.cc:428] Detected that this master 9a0da90f06dc41838c0eade213130e3a is joining an existing cluster
I20260812 03:57:53.863222 7274 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.43.252:34789
I20260812 03:57:53.872525 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.864442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48628 (local address 127.31.43.254:35427)
0812 03:57:53.864489 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.864491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.864661 (+ 170us) server_negotiation.cc:407] Connection header received
0812 03:57:53.864780 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.864781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.864797 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.864830 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.865419 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.866248 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.867718 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.868013 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.868054 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.869312 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.869314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.869314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.869323 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.871065 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.871453 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.871454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.871454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.871564 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.871845 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.871847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.871847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.872010 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.872088 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.872300 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.872398 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:57:53.872519 7383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.864385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:48628)
0812 03:57:53.864595 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.864607 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.864692 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.864935 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.864936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.865210 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.865317 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.865320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.866346 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.866346 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.867626 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.867628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.868173 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.868174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.868234 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.868801 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.868806 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.869191 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.871165 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.871166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.871166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.871366 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.871683 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.871684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.871685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.871765 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.872107 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.872109 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.872173 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.872247 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.872373 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":137,"thread_start_us":180,"threads_started":1}
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507072, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.254@KRBTEST.COM for oryx/127.31.43.252@KRBTEST.COM
I20260812 03:57:53.885001 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 6579
I20260812 03:57:53.904258 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.864749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:32979 (local address 127.0.0.1:46936)
0812 03:57:53.864982 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.864992 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.865082 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.865273 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.865273 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.865513 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.865589 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.865591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.866491 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.866491 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.867512 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.867516 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.868165 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.868166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.868234 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.899887 (+ 31653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.899895 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.900812 (+ 917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.903047 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.903048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.903049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.903272 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.903518 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.903519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.903519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.903577 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.903889 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.903890 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.903939 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.904003 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.904131 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":66,"thread_start_us":146,"threads_started":1}
I20260812 03:57:53.904578 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.864742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46936 (local address 127.31.43.253:32979)
0812 03:57:53.864785 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.864787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.865044 (+ 257us) server_negotiation.cc:407] Connection header received
0812 03:57:53.865158 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.865159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.865171 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.865204 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.865683 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.866401 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.867627 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.867933 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.867985 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.901110 (+ 33125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.901113 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.901113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.901126 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.902889 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.903382 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.903384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.903384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.903447 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.903656 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.903656 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.903657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.903802 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.903863 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.904402 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.904486 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:57:53.906764 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:37083
--local_ip_for_outbound_sockets=127.31.43.193
--tserver_master_addrs=127.31.43.254:35427
--webserver_port=41963
--webserver_interface=127.31.43.193
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab
--principal=oryx/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.43.254:35427,127.31.43.253:32979
--tserver_master_addrs=127.31.43.254:35427,127.31.43.253:32979,127.31.43.252:34789 with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:57:53.907227 7356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.864649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35180 (local address 127.31.43.252:34789)
0812 03:57:53.864693 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.864695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.865038 (+ 343us) server_negotiation.cc:407] Connection header received
0812 03:57:53.865145 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.865147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.865156 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.865183 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.865685 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.866381 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.867711 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.868015 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.868059 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.903768 (+ 35709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.903769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.903769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.903777 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.905930 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.906370 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.906372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.906372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.906440 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.906647 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.906648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.906648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.906810 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.906853 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.907053 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.907135 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:57:53.907533 7386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.864432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34789 (local address 127.0.0.1:35180)
0812 03:57:53.864969 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.864982 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.865076 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.865240 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.865241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.865496 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.865597 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.865600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.866470 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.866470 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.867614 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.867618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.868153 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.868154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.868213 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.898844 (+ 30631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.898853 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.902595 (+ 3742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.906055 (+ 3460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.906056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.906056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.906288 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.906520 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.906520 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.906521 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.906574 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.906894 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.906895 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.906930 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.907334 (+ 404us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.907434 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260812 03:57:53.907721 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.875432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34789 (local address 127.0.0.1:35184)
0812 03:57:53.875496 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.875508 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.875551 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.875802 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.875803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.876090 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.876144 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.876147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.877038 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.877039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.877573 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.877575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.878007 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.878008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.878082 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.878560 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.878566 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.880198 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.906414 (+ 26216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.906416 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.906417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.906700 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.906976 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.906977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.906978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.907048 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.907339 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.907340 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.907395 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.907444 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.907577 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:57:53.908222 7387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.875500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35184 (local address 127.31.43.252:34789)
0812 03:57:53.875670 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.875672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.875678 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:57:53.875711 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.875711 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.875721 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.875798 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.876247 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.876934 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.877662 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.877893 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.877980 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.880339 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.880341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.880341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.880350 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.903639 (+ 23289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.906798 (+ 3159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.906799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.906799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.906861 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.907125 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.907125 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.907126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.907263 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.907897 (+ 634us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.907965 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.908067 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":111,"threads_started":1}
I20260812 03:57:53.908680 6512 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.43.252:34789: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9a0da90f06dc41838c0eade213130e3a"
member_type: NON_VOTER
last_known_addr {
host: "127.31.43.252"
port: 34789
}
attrs {
promote: true
}
}
dest_uuid: "52dbb396108240fd9ce5b7b34010b505"
cas_config_opid_index: 5
I20260812 03:57:53.909017 6512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } }
I20260812 03:57:53.909577 7166 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9a0da90f06dc41838c0eade213130e3a (127.31.43.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:53.910859 7166 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.43.252:34789
I20260812 03:57:53.910959 7166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:53.911054 7166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:53.911535 7274 master.cc:537] Master@127.31.43.252:34789 shutting down...
I20260812 03:57:53.911695 7325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:53.912854 7325 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 1 FOLLOWER]: Refusing update from remote peer 52dbb396108240fd9ce5b7b34010b505: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 03:57:53.913242 7167 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:57:53.914111 6496 consensus_peers.cc:487] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 -> Peer 9a0da90f06dc41838c0eade213130e3a (127.31.43.252:34789): Couldn't send request to peer 9a0da90f06dc41838c0eade213130e3a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:57:53.915942 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.911267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:32979 (local address 127.0.0.1:46948)
0812 03:57:53.911534 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.911547 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.911594 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.911936 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.911937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.911950 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:53.912009 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.912012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.913195 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.913197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.914513 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.914518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.915614 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.915615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.915690 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.915727 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.915784 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.915842 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248}
I20260812 03:57:53.916266 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.911344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46948 (local address 127.31.43.253:32979)
0812 03:57:53.911402 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.911404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.911647 (+ 243us) server_negotiation.cc:407] Connection header received
0812 03:57:53.911678 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.911679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.911707 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.911769 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:53.912123 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.913004 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.914636 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.915394 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.915473 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.915563 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.916115 (+ 552us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.916174 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:57:53.916379 7274 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:53.916487 7274 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:53.916515 7224 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 LEARNER]: Refusing update from remote peer 52dbb396108240fd9ce5b7b34010b505: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 03:57:53.916549 7274 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: stopping tablet replica
I20260812 03:57:53.916781 7389 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 03:57:53.917439 7224 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9a0da90f06dc41838c0eade213130e3a (127.31.43.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:53.919097 7258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52dbb396108240fd9ce5b7b34010b505. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:53.919202 7258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:57:53.919260 7258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: NON_VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: true } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:53.919312 7258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:57:53.919726 7389 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505: attempting to promote NON_VOTER 32f35d95da9e45b78a40bacdc04b6c53 to VOTER
I20260812 03:57:53.920001 7389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } }
I20260812 03:57:53.920538 7224 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 LEARNER]: Refusing update from remote peer 52dbb396108240fd9ce5b7b34010b505: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 03:57:53.920572 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.920443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34789 (local address 127.0.0.1:35188)
0812 03:57:53.920486 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.920534 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 03:57:53.920789 7167 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 03:57:53.921027 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.920838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34789 (local address 127.0.0.1:35202)
0812 03:57:53.920968 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.920995 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
W20260812 03:57:53.921079 6496 consensus_peers.cc:487] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 -> Peer 9a0da90f06dc41838c0eade213130e3a (127.31.43.252:34789): Couldn't send request to peer 9a0da90f06dc41838c0eade213130e3a. Status: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 03:57:53.921491 7167 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 32f35d95da9e45b78a40bacdc04b6c53 (127.31.43.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:53.922448 7167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:53.922549 7167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:53.922611 7167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:53.922662 7167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:53.922957 7224 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 32f35d95da9e45b78a40bacdc04b6c53 (127.31.43.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:53.923905 7391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:53.924018 7391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:53.930629 7274 master.cc:552] Master@127.31.43.252:34789 shutdown complete.
I20260812 03:57:53.930738 7274 master_runner.cc:305] Clearing existing system tablet
I20260812 03:57:53.930882 7274 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:53.930949 7274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:53.931643 7274 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:57:53.932194 7395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.932325 7274 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:53.932386 7274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:57:53.932417 7274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/wal
uuid: "9a0da90f06dc41838c0eade213130e3a"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "7b67f70669e2c1f8a5524c116fc10ded"
server_key_iv: "5035497fcf71102421900482dc0f33bf"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.933192 7274 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:53.934919 7274 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 03:57:53.935043 7274 log.cc:1196] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 03:57:53.935312 7274 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: Deleting consensus metadata
I20260812 03:57:53.935458 7274 master_runner.cc:315] Copying system tablet from 127.31.43.254:35427
I20260812 03:57:53.936182 7274 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: tablet copy: Beginning tablet copy session from remote peer at address 127.31.43.254:35427
I20260812 03:57:53.942971 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.936620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48634 (local address 127.31.43.254:35427)
0812 03:57:53.936675 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:53.936677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:53.936816 (+ 139us) server_negotiation.cc:407] Connection header received
0812 03:57:53.936887 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:53.936888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:53.936901 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:53.936961 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:53.937342 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.937859 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.938613 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.938894 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.938974 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.940076 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:53.940078 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:53.940078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:53.940087 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:53.941620 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.942041 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.942043 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.942043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.942107 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:53.942322 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:53.942323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:53.942323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:53.942444 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:53.942525 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:53.942808 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 03:57:53.942887 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:57:53.943015 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:53.936577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:48634)
0812 03:57:53.936727 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:53.936739 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:53.936814 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:53.936968 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:53.936968 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:53.937189 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:53.937251 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.937253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.937948 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.937950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.938517 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:53.938519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:53.938977 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:53.938978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:53.939052 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:53.939637 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:53.939642 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:53.939973 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:53.941750 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.941752 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.941752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.941951 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.942183 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:53.942183 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:53.942184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:53.942250 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:53.942515 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:53.942516 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:53.942569 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 03:57:53.942662 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 03:57:53.942849 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260812 03:57:53.943472 6532 tablet_copy_service.cc:140] P 52dbb396108240fd9ce5b7b34010b505: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9a0da90f06dc41838c0eade213130e3a ({username='oryx', principal='oryx/127.31.43.252@KRBTEST.COM'} at 127.0.0.1:48634)
I20260812 03:57:53.943568 6532 tablet_copy_service.cc:161] P 52dbb396108240fd9ce5b7b34010b505: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9a0da90f06dc41838c0eade213130e3a at {username='oryx', principal='oryx/127.31.43.252@KRBTEST.COM'} at 127.0.0.1:48634: session id = 9a0da90f06dc41838c0eade213130e3a-00000000000000000000000000000000
I20260812 03:57:53.944096 6532 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:57:53.944717 7274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 03:57:53.945904 7274 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: tablet copy: Starting download of 0 data blocks...
I20260812 03:57:53.946089 7274 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: tablet copy: Starting download of 1 WAL segments...
I20260812 03:57:53.947419 7274 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:57:53.948284 6532 tablet_copy_service.cc:342] P 52dbb396108240fd9ce5b7b34010b505: Request end of tablet copy session 9a0da90f06dc41838c0eade213130e3a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.43.252@KRBTEST.COM'} at 127.0.0.1:48634
I20260812 03:57:53.948370 6532 tablet_copy_service.cc:433] P 52dbb396108240fd9ce5b7b34010b505: ending tablet copy session 9a0da90f06dc41838c0eade213130e3a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9a0da90f06dc41838c0eade213130e3a
I20260812 03:57:53.950908 7274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:53.952534 7408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:53.952764 7274 server_base.cc:1034] running on GCE node
W20260812 03:57:53.952898 7404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:53.955835 7405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:53.956028 7274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:53.956243 7274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:53.957391 7274 hybrid_clock.cc:648] HybridClock initialized: now 1786507073957377 us; error 28 us; skew 500 ppm
Aug 12 03:57:53 dist-test-slave-8tdl krb5kdc[6462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507073, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:53.959066 7274 init.cc:377] Logged in from keytab as oryx/127.31.43.252@KRBTEST.COM (short username oryx)
I20260812 03:57:53.959451 7274 webserver.cc:460] Webserver started at http://127.31.43.252:45091/ using document root <none> and password file <none>
I20260812 03:57:53.959584 7274 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:53.959625 7274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:53.960417 7274 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:53.961028 7415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:53.961174 7274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:57:53.961225 7274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/wal
uuid: "9a0da90f06dc41838c0eade213130e3a"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "7b67f70669e2c1f8a5524c116fc10ded"
server_key_iv: "5035497fcf71102421900482dc0f33bf"
server_key_version: "encryptionkey@0"
I20260812 03:57:53.961297 7274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:53.969740 7274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:53.971580 7274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:53.975028 7274 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.252:34789
I20260812 03:57:53.975455 7470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.252:34789 every 8 connection(s)
I20260812 03:57:53.975476 7274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
I20260812 03:57:53.976456 7471 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:57:53.976915 7471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: Bootstrap starting.
I20260812 03:57:53.979461 7471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:57:53.979717 7471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: Bootstrap complete.
I20260812 03:57:53.979964 7471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } }
I20260812 03:57:53.980063 7471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a0da90f06dc41838c0eade213130e3a, State: Initialized, Role: LEARNER
I20260812 03:57:53.980108 7471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } }
I20260812 03:57:53.980314 7471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:53.980299 7476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:53.980528 7476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: This master's current role is: LEARNER
I20260812 03:57:53.982220 7490 catalog_manager.cc:1259] Loaded cluster ID: 9eab5de793e84a0fb81fbd2348d322b7
I20260812 03:57:53.982313 7490 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: loading cluster ID for follower catalog manager: success
I20260812 03:57:53.983507 7490 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: acquiring CA information for follower catalog manager: success
I20260812 03:57:53.983888 7490 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 03:57:54.040627 7388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:54.040943 7388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:54.040990 7388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:54.042961 7388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:54.043053 7388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:54.043103 7388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:54.045693 7388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab
--principal=oryx/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:37083
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=41963
--tserver_master_addrs=127.31.43.254:35427,127.31.43.253:32979,127.31.43.252:34789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:54.046015 7388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:54.046336 7388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:54.052591 7496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:54.052701 7388 server_base.cc:1034] running on GCE node
W20260812 03:57:54.052839 7498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:54.052591 7495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:54.053172 7388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:54.053483 7388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:54.054653 7388 hybrid_clock.cc:648] HybridClock initialized: now 1786507074054642 us; error 38 us; skew 500 ppm
Aug 12 03:57:54 dist-test-slave-8tdl krb5kdc[6462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507074, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:54.057513 7388 init.cc:377] Logged in from keytab as oryx/127.31.43.193@KRBTEST.COM (short username oryx)
I20260812 03:57:54.058082 7388 webserver.cc:460] Webserver started at http://127.31.43.193:41963/ using document root <none> and password file <none>
I20260812 03:57:54.058353 7388 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:54.058444 7388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:54.060163 7388 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:54.060976 7507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:54.061219 7388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:54.061342 7388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "bf97a7334d184009840330be17a07833"
format_stamp: "Formatted at 2026-08-12 03:57:52 on dist-test-slave-8tdl"
server_key: "61aada7451bf1a53dfc37ea1f792f6d7"
server_key_iv: "e3c70d43312d10f5fc54ac110d003056"
server_key_version: "encryptionkey@0"
I20260812 03:57:54.061787 7388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:54.082321 7388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:54.085034 7388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:54.085247 7388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:54.085506 7388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:54.085810 7388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:54.085842 7388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:54.085866 7388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:54.085878 7388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:54.095429 7388 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:37083
I20260812 03:57:54.095741 7637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:37083 every 8 connection(s)
I20260812 03:57:54.095934 7388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:57:54 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507074, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.252@KRBTEST.COM
Aug 12 03:57:54 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507074, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.253@KRBTEST.COM
Aug 12 03:57:54 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507074, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:54.105931 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 7388
I20260812 03:57:54.106140 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 6735
I20260812 03:57:54.110028 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.097001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:40781 (local address 127.31.43.254:35427)
0812 03:57:54.097085 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.097087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.097316 (+ 229us) server_negotiation.cc:407] Connection header received
0812 03:57:54.098622 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.098623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.098639 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.098726 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.099639 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.100486 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.101714 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.101988 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.102056 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.106137 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.106138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.106139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.106148 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.108066 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.108684 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.108686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.108686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.108753 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.109117 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.109118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.109119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.109287 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.109341 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.109801 (+ 460us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.109899 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:57:54.110733 7646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.096935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.193:40781)
0812 03:57:54.097156 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.097205 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.098461 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.098814 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.098818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.099294 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:54.099527 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.099532 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.100620 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.100621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.101637 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.101640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.102086 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.102087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.102247 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.102974 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:54.102983 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:54.106025 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:54.108173 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.108187 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.108189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.108580 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.108837 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.108838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.108838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.109022 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.109394 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:54.109401 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:54.109615 (+ 214us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.110202 (+ 587us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.110439 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":48,"thread_start_us":103,"threads_started":1}
I20260812 03:57:54.111162 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.096945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:47081 (local address 127.31.43.252:34789)
0812 03:57:54.097262 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.097264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.097685 (+ 421us) server_negotiation.cc:407] Connection header received
0812 03:57:54.098666 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.098668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.098710 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.098753 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.099772 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.100549 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.101904 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.102211 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.102310 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.106067 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.106069 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.106070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.106081 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.109786 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.110193 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.110195 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.110195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.110265 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.110499 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.110500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.110500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.110639 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.110704 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.110936 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.111045 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":65,"threads_started":1}
I20260812 03:57:54.111258 7639 heartbeater.cc:347] Connected to a master server at 127.31.43.254:35427
I20260812 03:57:54.111351 7639 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:54.111477 7647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.096935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34789 (local address 127.31.43.193:47081)
0812 03:57:54.097580 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.097591 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.098535 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.098835 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.098836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.099353 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:54.099704 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.099706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.100674 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.100674 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.101821 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.101823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.102316 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.102318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.102397 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.103211 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:54.103217 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:54.105659 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:54.109876 (+ 4217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.109877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.109878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.110092 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.110342 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.110342 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.110342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.110404 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.110756 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:54.110757 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:54.110811 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.111250 (+ 439us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.111372 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":8,"thread_start_us":117,"threads_started":1}
I20260812 03:57:54.111560 7639 heartbeater.cc:511] Master 127.31.43.254:35427 requested a full tablet report, sending...
I20260812 03:57:54.112057 6512 ts_manager.cc:194] Re-registered known tserver with Master: bf97a7334d184009840330be17a07833 (127.31.43.193:37083)
I20260812 03:57:54.112396 6512 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:40781
I20260812 03:57:54.115592 7638 heartbeater.cc:347] Connected to a master server at 127.31.43.252:34789
I20260812 03:57:54.115674 7638 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:54.115782 7638 heartbeater.cc:511] Master 127.31.43.252:34789 requested a full tablet report, sending...
I20260812 03:57:54.116333 7433 ts_manager.cc:194] Registered new tserver with Master: bf97a7334d184009840330be17a07833 (127.31.43.193:37083)
I20260812 03:57:54.121201 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:33159
--local_ip_for_outbound_sockets=127.31.43.194
--tserver_master_addrs=127.31.43.254:35427
--webserver_port=39825
--webserver_interface=127.31.43.194
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab
--principal=oryx/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.43.254:35427,127.31.43.253:32979
--tserver_master_addrs=127.31.43.254:35427,127.31.43.253:32979,127.31.43.252:34789 with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:57:54.122076 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.097370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:56807 (local address 127.31.43.253:32979)
0812 03:57:54.097409 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.097411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.097864 (+ 453us) server_negotiation.cc:407] Connection header received
0812 03:57:54.098666 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.098667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.098682 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.098759 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.099736 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.100461 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.101867 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.102158 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.102204 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.105464 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.105466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.105466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.105475 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.120386 (+ 14911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.120936 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.120937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.120937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.121023 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.121404 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.121405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.121406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.121625 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.121691 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.121892 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.121986 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:57:54.122282 7644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.097288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:32979 (local address 127.31.43.193:56807)
0812 03:57:54.097781 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.097790 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.098535 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.098836 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.098837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.099352 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:54.099529 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.099532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.100647 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.100648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.101759 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.101762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.102294 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.102295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.102363 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.103154 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:54.103158 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:54.105314 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:54.120537 (+ 15223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.120540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.120540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.120836 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.121111 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.121112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.121112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.121189 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.121700 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:54.121701 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:54.121772 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.121855 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.122021 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":54,"thread_start_us":227,"threads_started":1}
I20260812 03:57:54.122804 7640 heartbeater.cc:347] Connected to a master server at 127.31.43.253:32979
I20260812 03:57:54.122874 7640 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:54.122946 7640 heartbeater.cc:511] Master 127.31.43.253:32979 requested a full tablet report, sending...
I20260812 03:57:54.123391 7212 ts_manager.cc:194] Registered new tserver with Master: bf97a7334d184009840330be17a07833 (127.31.43.193:37083)
W20260812 03:57:54.212229 7648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:54.212474 7648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:54.212503 7648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:54.213722 7648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:54.213768 7648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:54.213796 7648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:54.215525 7648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab
--principal=oryx/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:33159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=39825
--tserver_master_addrs=127.31.43.254:35427,127.31.43.253:32979,127.31.43.252:34789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:54.215775 7648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:54.215984 7648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:54.221778 7656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:54.221786 7654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:54.221827 7648 server_base.cc:1034] running on GCE node
W20260812 03:57:54.221786 7653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:54.222267 7648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:54.222548 7648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:54.223739 7648 hybrid_clock.cc:648] HybridClock initialized: now 1786507074223676 us; error 81 us; skew 500 ppm
Aug 12 03:57:54 dist-test-slave-8tdl krb5kdc[6462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507074, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:54.230011 7648 init.cc:377] Logged in from keytab as oryx/127.31.43.194@KRBTEST.COM (short username oryx)
I20260812 03:57:54.230556 7648 webserver.cc:460] Webserver started at http://127.31.43.194:39825/ using document root <none> and password file <none>
I20260812 03:57:54.230857 7648 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:54.230960 7648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:54.232517 7648 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:54.233273 7663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:54.233490 7648 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:54.233603 7648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "325b8f3915404001925a596ef97ed930"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "741249e5327a4304b35aa4ecd437d294"
server_key_iv: "9f8346e032b1bdad1ed67579be1a4949"
server_key_version: "encryptionkey@0"
I20260812 03:57:54.234014 7648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:54.246093 7648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:54.248584 7648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:54.248771 7648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:54.249044 7648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:54.249405 7648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:54.249464 7648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:54.249509 7648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:54.249540 7648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:54.255833 7648 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:33159
I20260812 03:57:54.255925 7797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:33159 every 8 connection(s)
I20260812 03:57:54.256336 7648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:57:54.258066 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 7648
I20260812 03:57:54.258255 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 6890
Aug 12 03:57:54 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507074, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
Aug 12 03:57:54 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507074, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.252@KRBTEST.COM
Aug 12 03:57:54 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507074, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.253@KRBTEST.COM
I20260812 03:57:54.274453 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.257189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:60011 (local address 127.31.43.253:32979)
0812 03:57:54.257245 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.257248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.257749 (+ 501us) server_negotiation.cc:407] Connection header received
0812 03:57:54.261673 (+ 3924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.261675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.261694 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.261730 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.263250 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.264101 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.266011 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.266337 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.266381 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.271182 (+ 4801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.271183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.271184 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.271192 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.272999 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.273394 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.273395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.273395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.273453 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.273655 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.273656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.273656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.273782 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.273837 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.274251 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.274348 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:57:54.274458 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.257253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:39133 (local address 127.31.43.252:34789)
0812 03:57:54.257299 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.257300 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.257571 (+ 271us) server_negotiation.cc:407] Connection header received
0812 03:57:54.262183 (+ 4612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.262185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.262205 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.262240 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.263286 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.264349 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.266197 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.266642 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.266744 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.270818 (+ 4074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.270819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.270820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.270829 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.272664 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.273244 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.273245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.273246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.273308 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.273627 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.273628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.273628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.273782 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.273837 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.274251 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.274348 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:57:54.275167 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.257198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34789 (local address 127.31.43.194:39133)
0812 03:57:54.257425 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.257471 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.261491 (+ 4020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.262310 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.262311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.262644 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:54.263106 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.263122 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.264480 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.264482 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.265978 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.265983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.266852 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.266854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.266954 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.267605 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:54.267611 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:54.270718 (+ 3107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:54.272774 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.272785 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.272788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.273150 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.273377 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.273377 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.273378 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.273543 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.273876 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:54.273887 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:54.274079 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.274637 (+ 558us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.274915 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":77,"thread_start_us":114,"threads_started":1}
I20260812 03:57:54.275456 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.257632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:39365 (local address 127.31.43.254:35427)
0812 03:57:54.257697 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.257699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.257831 (+ 132us) server_negotiation.cc:407] Connection header received
0812 03:57:54.261722 (+ 3891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.261724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.261739 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.261771 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.263250 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.264050 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.266012 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.266332 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.266381 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.270560 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.270563 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.270563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.270575 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.273663 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.274234 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.274236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.274237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.274315 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.274583 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.274584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.274584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.274800 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.274890 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.275181 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.275297 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:57:54.275712 7805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.257198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:32979 (local address 127.31.43.194:60011)
0812 03:57:54.257677 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.257688 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.261524 (+ 3836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.261992 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.261997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.262614 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:54.263105 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.263114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.264243 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.264244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.265869 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.265873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.266509 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.266510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.266722 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.267456 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:54.267469 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:54.271024 (+ 3555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:54.273093 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.273094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.273095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.273303 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.273522 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.273523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.273523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.273579 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.273876 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:54.273887 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:54.274079 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.275072 (+ 993us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.275186 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":204,"thread_start_us":103,"threads_started":1}
I20260812 03:57:54.275952 7804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.257554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.194:39365)
0812 03:57:54.257773 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.257781 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.261622 (+ 3841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.261992 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.261997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.262691 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:54.263105 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.263114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.264243 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.264244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.265869 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.265873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.266509 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.266510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.266722 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.267733 (+ 1011us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:54.267739 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:54.270425 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:54.273787 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.273789 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.273789 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.274104 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.274406 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.274407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.274408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.274490 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.274935 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:54.274936 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:54.274997 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.275626 (+ 629us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.275778 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":122,"thread_start_us":76,"threads_started":1}
I20260812 03:57:54.275712 7798 heartbeater.cc:347] Connected to a master server at 127.31.43.252:34789
I20260812 03:57:54.276104 7798 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:54.276378 7798 heartbeater.cc:511] Master 127.31.43.252:34789 requested a full tablet report, sending...
I20260812 03:57:54.276618 7800 heartbeater.cc:347] Connected to a master server at 127.31.43.253:32979
I20260812 03:57:54.276705 7800 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:54.276818 7800 heartbeater.cc:511] Master 127.31.43.253:32979 requested a full tablet report, sending...
I20260812 03:57:54.276964 7799 heartbeater.cc:347] Connected to a master server at 127.31.43.254:35427
I20260812 03:57:54.277037 7799 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:54.277068 7433 ts_manager.cc:194] Registered new tserver with Master: 325b8f3915404001925a596ef97ed930 (127.31.43.194:33159)
I20260812 03:57:54.277140 7799 heartbeater.cc:511] Master 127.31.43.254:35427 requested a full tablet report, sending...
I20260812 03:57:54.277398 7212 ts_manager.cc:194] Registered new tserver with Master: 325b8f3915404001925a596ef97ed930 (127.31.43.194:33159)
I20260812 03:57:54.277810 6512 ts_manager.cc:194] Re-registered known tserver with Master: 325b8f3915404001925a596ef97ed930 (127.31.43.194:33159)
I20260812 03:57:54.278530 6512 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:39365
I20260812 03:57:54.284238 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:38991
--local_ip_for_outbound_sockets=127.31.43.195
--tserver_master_addrs=127.31.43.254:35427
--webserver_port=40727
--webserver_interface=127.31.43.195
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab
--principal=oryx/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.43.254:35427,127.31.43.253:32979
--tserver_master_addrs=127.31.43.254:35427,127.31.43.253:32979,127.31.43.252:34789 with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:57:54.341210 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.338270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50534 (local address 127.31.43.252:34789)
0812 03:57:54.338359 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.338361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.338374 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:57:54.338414 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.338415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.338465 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.338552 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:54.338770 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.339280 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.340361 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.340830 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.340927 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.341011 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.341080 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.341123 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:57:54.341212 7392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.338189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34789 (local address 127.0.0.1:50534)
0812 03:57:54.338270 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.338283 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.338357 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.338562 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.338562 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.338566 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:54.338645 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.338647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.339379 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.339380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.340249 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.340252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.340930 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.340931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.341012 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.341017 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.341076 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.341130 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 03:57:54.341614 7444 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 03:57:54.341885 7476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52dbb396108240fd9ce5b7b34010b505. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:54.342037 7476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: This master's current role is: LEARNER
W20260812 03:57:54.376416 7806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:54.376770 7806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:54.376819 7806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:54.378144 7806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:54.378229 7806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:54.378278 7806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:54.380144 7806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab
--principal=oryx/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:38991
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=40727
--tserver_master_addrs=127.31.43.254:35427,127.31.43.253:32979,127.31.43.252:34789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:54.380486 7806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:54.380779 7806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:54.386942 7814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:54.386942 7811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:54.386942 7812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:54.387323 7806 server_base.cc:1034] running on GCE node
I20260812 03:57:54.387620 7806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:54.387846 7806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:54.389107 7806 hybrid_clock.cc:648] HybridClock initialized: now 1786507074389039 us; error 41 us; skew 500 ppm
Aug 12 03:57:54 dist-test-slave-8tdl krb5kdc[6462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507074, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:54.391999 7806 init.cc:377] Logged in from keytab as oryx/127.31.43.195@KRBTEST.COM (short username oryx)
I20260812 03:57:54.392459 7806 webserver.cc:460] Webserver started at http://127.31.43.195:40727/ using document root <none> and password file <none>
I20260812 03:57:54.392659 7806 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:54.392715 7806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:54.394230 7806 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:54.394908 7823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:54.395093 7806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:54.395197 7806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "76634ab8eb914593903c21a6307a1a7e"
format_stamp: "Formatted at 2026-08-12 03:57:53 on dist-test-slave-8tdl"
server_key: "5dd8589da9c103694dda3951cb0e8b53"
server_key_iv: "c05616577278b0e340df28a9fb8dcfd4"
server_key_version: "encryptionkey@0"
I20260812 03:57:54.395557 7806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:54.404361 7806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:54.406836 7806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:54.406965 7806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:54.407176 7806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:57:54.407524 7806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:54.407562 7806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:54.407615 7806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:54.407641 7806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:54.413813 7806 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:38991
I20260812 03:57:54.413934 7952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:38991 every 8 connection(s)
I20260812 03:57:54.414263 7806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:57:54.421178 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 7806
Aug 12 03:57:54 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507074, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.252@KRBTEST.COM
Aug 12 03:57:54 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507074, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.253@KRBTEST.COM
Aug 12 03:57:54 dist-test-slave-8tdl krb5kdc[6462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507074, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:54.429253 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.415179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:36583 (local address 127.31.43.254:35427)
0812 03:57:54.415236 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.415239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.415588 (+ 349us) server_negotiation.cc:407] Connection header received
0812 03:57:54.416330 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.416332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.416361 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.416437 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.417702 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.418239 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.420571 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.420861 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.420936 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.425061 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.425063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.425063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.425071 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.427320 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.428003 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.428004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.428005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.428072 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.428373 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.428374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.428374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.428514 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.428561 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.429030 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.429127 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:57:54.429757 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.415649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:45741 (local address 127.31.43.253:32979)
0812 03:57:54.415707 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.415710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.415982 (+ 272us) server_negotiation.cc:407] Connection header received
0812 03:57:54.416365 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.416366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.416379 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.416413 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.417692 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.418307 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.420064 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.420397 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.420470 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.425168 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.425170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.425170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.425179 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.427569 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.428919 (+ 1350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.428920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.428920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.428986 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.429199 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.429200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.429200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.429325 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.429370 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.429598 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.429673 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:57:54.429910 7957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.415181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.195:36583)
0812 03:57:54.415451 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.415499 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.416134 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.416524 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.416524 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.417274 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:54.417582 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.417585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.418367 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.418368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.419959 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.419962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.421509 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.421510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.421585 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.422011 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:54.422017 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:54.424920 (+ 2903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:54.427428 (+ 2508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.427441 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.427444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.427889 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.428149 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.428150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.428150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.428302 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.428611 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:54.428619 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:54.428820 (+ 201us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.429428 (+ 608us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.429647 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":47,"thread_start_us":123,"threads_started":1}
I20260812 03:57:54.430099 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.415551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:32979 (local address 127.31.43.195:45741)
0812 03:57:54.415701 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.415712 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.416134 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.416498 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.416499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.417224 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:54.417562 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.417571 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.418414 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.418414 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.419959 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.419962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.420506 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.420508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.420742 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.421515 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:54.421530 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:54.425069 (+ 3539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:54.427659 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.427660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.427660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.428826 (+ 1166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.429066 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.429067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.429067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.429128 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.429424 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:54.429425 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:54.429469 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.429888 (+ 419us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.430000 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":112,"thread_start_us":70,"threads_started":1}
I20260812 03:57:54.430596 7955 heartbeater.cc:347] Connected to a master server at 127.31.43.253:32979
I20260812 03:57:54.430760 7955 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:54.430976 7954 heartbeater.cc:347] Connected to a master server at 127.31.43.254:35427
I20260812 03:57:54.431002 7955 heartbeater.cc:511] Master 127.31.43.253:32979 requested a full tablet report, sending...
I20260812 03:57:54.431052 7954 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:54.431149 7954 heartbeater.cc:511] Master 127.31.43.254:35427 requested a full tablet report, sending...
I20260812 03:57:54.431576 7212 ts_manager.cc:194] Registered new tserver with Master: 76634ab8eb914593903c21a6307a1a7e (127.31.43.195:38991)
I20260812 03:57:54.431577 6512 ts_manager.cc:194] Re-registered known tserver with Master: 76634ab8eb914593903c21a6307a1a7e (127.31.43.195:38991)
I20260812 03:57:54.432060 6512 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:36583
I20260812 03:57:54.432157 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.415290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:52435 (local address 127.31.43.252:34789)
0812 03:57:54.415370 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.415372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.415629 (+ 257us) server_negotiation.cc:407] Connection header received
0812 03:57:54.416204 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.416206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.416223 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.416277 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.418341 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.418927 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.420137 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.420453 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.420614 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.425028 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.425030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.425030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.425039 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.428170 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.430312 (+ 2142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.430314 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.430315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.430397 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.430657 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.430658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.430659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.430862 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.430939 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.431923 (+ 984us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.432065 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:57:54.432890 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.415181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34789 (local address 127.31.43.195:52435)
0812 03:57:54.415471 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.415501 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.416092 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.416372 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.416375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.417257 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:54.417563 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.417571 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.419053 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.419054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.419955 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.419958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.420565 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.420566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.420736 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.421521 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:54.421529 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:54.424894 (+ 3365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:54.428683 (+ 3789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.428684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.428685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.428971 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.430483 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:54.430484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:54.430484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:54.430568 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:54.431736 (+ 1168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:54.431737 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:54.431789 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.431871 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.432023 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":141,"thread_start_us":86,"threads_started":1}
I20260812 03:57:54.434342 7953 heartbeater.cc:347] Connected to a master server at 127.31.43.252:34789
I20260812 03:57:54.434473 7953 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:54.434617 7953 heartbeater.cc:511] Master 127.31.43.252:34789 requested a full tablet report, sending...
I20260812 03:57:54.435149 7433 ts_manager.cc:194] Registered new tserver with Master: 76634ab8eb914593903c21a6307a1a7e (127.31.43.195:38991)
I20260812 03:57:54.436070 7965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.422587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57660 (local address 127.31.43.254:35427)
0812 03:57:54.423642 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.423645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.423653 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:57:54.423689 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.423689 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.423699 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.423758 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.424749 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.425632 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.427139 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.427531 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.427613 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.431712 (+ 4099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.431713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.431714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.431723 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.433838 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.435104 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.435106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.435106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.435184 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.435456 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.435457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.435457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.435586 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.435714 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.435837 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.435951 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1032,"thread_start_us":67,"threads_started":1}
I20260812 03:57:54.438727 7966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.423031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.31.43.253:32979)
0812 03:57:54.423898 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.423902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.423910 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:57:54.423939 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.423940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.423951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.424171 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.426330 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.428428 (+ 2098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.429826 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.432191 (+ 2365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.432703 (+ 512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.435325 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.435327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.435327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.435337 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.437226 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.437679 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.437681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.437681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.437934 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.438121 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.438121 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.438122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.438245 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.438384 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.438498 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.438585 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":844,"thread_start_us":75,"threads_started":1}
I20260812 03:57:54.438263 6512 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:54.440428 6512 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:54.441452 7389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: NON_VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: true } } }
I20260812 03:57:54.441569 7389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:54.443853 7389 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505: attempting to promote NON_VOTER 9a0da90f06dc41838c0eade213130e3a to VOTER
I20260812 03:57:54.444016 7389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } }
I20260812 03:57:54.444622 7442 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 1 LEARNER]: Refusing update from remote peer 52dbb396108240fd9ce5b7b34010b505: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 03:57:54.444975 7224 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 FOLLOWER]: Refusing update from remote peer 52dbb396108240fd9ce5b7b34010b505: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 03:57:54.445264 7979 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 03:57:54.445914 7971 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 03:57:54.446146 7979 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9a0da90f06dc41838c0eade213130e3a (127.31.43.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } } }
I20260812 03:57:54.446278 7224 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9a0da90f06dc41838c0eade213130e3a (127.31.43.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } } }
I20260812 03:57:54.447185 7979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } } }
I20260812 03:57:54.447285 7979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:54.447253 7389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } } }
I20260812 03:57:54.447317 7389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:54.447775 7969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.423403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50542 (local address 127.31.43.252:34789)
0812 03:57:54.424601 (+ 1198us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.424604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.424614 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:54.424641 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.424642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.424656 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.427691 (+ 3035us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.427742 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.428568 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.435711 (+ 7143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.436076 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.436157 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.441007 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.441010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.441010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.441020 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.445006 (+ 3986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.445740 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.445742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.445742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.445826 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.446129 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.446131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.446131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.446538 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.447469 (+ 931us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.447523 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.447637 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1170,"thread_start_us":62,"threads_started":1}
I20260812 03:57:54.449038 7972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } } }
I20260812 03:57:54.449143 7972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:54.449122 7980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } } }
I20260812 03:57:54.449249 7980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:54.449891 7444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9a0da90f06dc41838c0eade213130e3a (127.31.43.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } } }
I20260812 03:57:54.451725 7476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52dbb396108240fd9ce5b7b34010b505" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } } }
I20260812 03:57:54.451831 7476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:54.454403 7983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.449365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:33159 (local address 127.0.0.1:41828)
0812 03:57:54.449653 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.449666 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.449720 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.450379 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.450380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.450383 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:54.450449 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.450451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.451563 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.451564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.452802 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.452805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.454118 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.454119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.454183 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.454188 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.454259 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.454305 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261}
I20260812 03:57:54.454424 7986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.449928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41828 (local address 127.31.43.194:33159)
0812 03:57:54.450174 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.450179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.450190 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:57:54.450232 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.450234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.450270 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.450375 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:54.450511 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.451475 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.452913 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.454019 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.454106 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.454195 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.454279 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.454329 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":124,"threads_started":1}
I20260812 03:57:54.454988 7392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.449443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:38991 (local address 127.0.0.1:47020)
0812 03:57:54.449653 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.449667 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.449719 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.450995 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.450996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.450998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:54.451036 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.451037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.452465 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.452466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.453555 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.453558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.454604 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.454606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.454705 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.454710 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.454763 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.454854 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":109,"threads_started":1}
I20260812 03:57:54.455209 7987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.450179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47020 (local address 127.31.43.195:38991)
0812 03:57:54.450667 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.450672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.450749 (+ 77us) server_negotiation.cc:407] Connection header received
0812 03:57:54.450797 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.450800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.450818 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.450918 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:54.451120 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.452337 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.453665 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.454480 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.454600 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.454973 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.455024 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.455084 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":208,"threads_started":1}
I20260812 03:57:54.455804 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.449490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:37083 (local address 127.0.0.1:45012)
0812 03:57:54.452102 (+ 2612us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.452117 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.452188 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.452406 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.452407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.452409 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:54.452465 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.452466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.453580 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.453583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.454416 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.454419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.455558 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.455559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.455632 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.455636 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.455685 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.455734 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2592,"thread_start_us":115,"threads_started":1}
I20260812 03:57:54.455935 7717 tablet_service.cc:1467] Processing CreateTablet for tablet cd348bcbe6594822a8fce339eca04905 (DEFAULT_TABLE table=test-table [id=4a598fe65ed748579e7454bcef37ad86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:54.456101 7984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.449606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45012 (local address 127.31.43.193:37083)
0812 03:57:54.449861 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.449874 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.452216 (+ 2342us) server_negotiation.cc:407] Connection header received
0812 03:57:54.452273 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.452276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.452294 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.452409 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:54.452555 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.453483 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.454528 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.455344 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.455396 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.455472 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.455942 (+ 470us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.455995 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":126,"threads_started":1}
I20260812 03:57:54.455935 7876 tablet_service.cc:1467] Processing CreateTablet for tablet cd348bcbe6594822a8fce339eca04905 (DEFAULT_TABLE table=test-table [id=4a598fe65ed748579e7454bcef37ad86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:54.456358 7876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd348bcbe6594822a8fce339eca04905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:54.456375 7717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd348bcbe6594822a8fce339eca04905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:54.456801 7565 tablet_service.cc:1467] Processing CreateTablet for tablet cd348bcbe6594822a8fce339eca04905 (DEFAULT_TABLE table=test-table [id=4a598fe65ed748579e7454bcef37ad86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:54.457186 7565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd348bcbe6594822a8fce339eca04905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:54.459934 7989 tablet_bootstrap.cc:492] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e: Bootstrap starting.
I20260812 03:57:54.461162 7989 tablet_bootstrap.cc:654] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:54.461517 7989 log.cc:824] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:54.461815 7990 tablet_bootstrap.cc:492] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833: Bootstrap starting.
I20260812 03:57:54.462442 7991 tablet_bootstrap.cc:492] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930: Bootstrap starting.
I20260812 03:57:54.462658 7989 tablet_bootstrap.cc:492] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e: No bootstrap required, opened a new log
I20260812 03:57:54.462769 7989 ts_tablet_manager.cc:1397] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260812 03:57:54.463120 7990 tablet_bootstrap.cc:654] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:54.463428 7991 tablet_bootstrap.cc:654] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:54.463474 7990 log.cc:824] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:54.463801 7991 log.cc:824] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:54.464526 7990 tablet_bootstrap.cc:492] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833: No bootstrap required, opened a new log
I20260812 03:57:54.464646 7990 ts_tablet_manager.cc:1397] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:57:54.464775 7991 tablet_bootstrap.cc:492] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930: No bootstrap required, opened a new log
I20260812 03:57:54.464869 7991 ts_tablet_manager.cc:1397] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:57:54.465595 7989 raft_consensus.cc:348] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:54.465801 7989 raft_consensus.cc:374] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:54.465840 7989 raft_consensus.cc:729] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76634ab8eb914593903c21a6307a1a7e, State: Initialized, Role: FOLLOWER
I20260812 03:57:54.465977 7989 consensus_queue.cc:260] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:54.466298 7989 ts_tablet_manager.cc:1428] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 03:57:54.466398 7954 heartbeater.cc:503] Master 127.31.43.254:35427 was elected leader, sending a full tablet report...
I20260812 03:57:54.468200 7990 raft_consensus.cc:348] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:54.468354 7990 raft_consensus.cc:374] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:54.468376 7990 raft_consensus.cc:729] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf97a7334d184009840330be17a07833, State: Initialized, Role: FOLLOWER
I20260812 03:57:54.468502 7990 consensus_queue.cc:260] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:54.468768 7990 ts_tablet_manager.cc:1428] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 03:57:54.468894 7639 heartbeater.cc:503] Master 127.31.43.254:35427 was elected leader, sending a full tablet report...
I20260812 03:57:54.468827 7991 raft_consensus.cc:348] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:54.469007 7991 raft_consensus.cc:374] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:54.469048 7991 raft_consensus.cc:729] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325b8f3915404001925a596ef97ed930, State: Initialized, Role: FOLLOWER
I20260812 03:57:54.469216 7991 consensus_queue.cc:260] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:54.470055 7991 ts_tablet_manager.cc:1428] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:57:54.470135 7799 heartbeater.cc:503] Master 127.31.43.254:35427 was elected leader, sending a full tablet report...
W20260812 03:57:54.507668 7801 tablet.cc:2369] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:57:54.556082 7998 raft_consensus.cc:482] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:54.556232 7998 raft_consensus.cc:504] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:54.556591 7998 leader_election.cc:288] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf97a7334d184009840330be17a07833 (127.31.43.193:37083), 76634ab8eb914593903c21a6307a1a7e (127.31.43.195:38991)
I20260812 03:57:54.560361 7987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.557026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:50599 (local address 127.31.43.195:38991)
0812 03:57:54.557080 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.557082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.557093 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:57:54.557153 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.557154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.557166 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.557244 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:54.557467 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.558001 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.559321 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.559921 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.560021 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.560031 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.560233 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.560280 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:57:54.560365 7805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.556826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:38991 (local address 127.31.43.194:50599)
0812 03:57:54.556960 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.556974 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.557030 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.557285 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.557287 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.557289 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:54.557386 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.557389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.558086 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.558087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.559206 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.559209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.560032 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.560033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.560144 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.560148 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.560215 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.560268 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":2}
I20260812 03:57:54.560647 7984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.556948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:38063 (local address 127.31.43.193:37083)
0812 03:57:54.557023 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.557026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.557266 (+ 240us) server_negotiation.cc:407] Connection header received
0812 03:57:54.557307 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.557309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.557318 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.557389 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:54.557562 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.558061 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.559314 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.559962 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.560025 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.560034 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.560283 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.560450 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:57:54.560772 7804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.556826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:37083 (local address 127.31.43.194:38063)
0812 03:57:54.557152 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:54.557164 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:54.557234 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:54.557398 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:54.557398 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:54.557399 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:54.557482 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.557484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.558162 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.558163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.559176 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:54.559199 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.560070 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.560071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:54.560185 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.560190 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:54.560353 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 03:57:54.560404 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":259}
I20260812 03:57:54.560976 7899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd348bcbe6594822a8fce339eca04905" candidate_uuid: "325b8f3915404001925a596ef97ed930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76634ab8eb914593903c21a6307a1a7e" is_pre_election: true
I20260812 03:57:54.561185 7899 raft_consensus.cc:2454] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 325b8f3915404001925a596ef97ed930 in term 0.
I20260812 03:57:54.561147 7585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd348bcbe6594822a8fce339eca04905" candidate_uuid: "325b8f3915404001925a596ef97ed930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf97a7334d184009840330be17a07833" is_pre_election: true
I20260812 03:57:54.561322 7585 raft_consensus.cc:2454] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 325b8f3915404001925a596ef97ed930 in term 0.
I20260812 03:57:54.561508 7668 leader_election.cc:302] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325b8f3915404001925a596ef97ed930, bf97a7334d184009840330be17a07833; no voters:
I20260812 03:57:54.561652 7998 raft_consensus.cc:2781] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:54.561746 7998 raft_consensus.cc:482] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:54.561777 7998 raft_consensus.cc:3037] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:54.563004 7998 raft_consensus.cc:504] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:54.563194 7998 leader_election.cc:288] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [CANDIDATE]: Term 1 election: Requested vote from peers bf97a7334d184009840330be17a07833 (127.31.43.193:37083), 76634ab8eb914593903c21a6307a1a7e (127.31.43.195:38991)
I20260812 03:57:54.563373 7585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd348bcbe6594822a8fce339eca04905" candidate_uuid: "325b8f3915404001925a596ef97ed930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf97a7334d184009840330be17a07833"
I20260812 03:57:54.563501 7585 raft_consensus.cc:3037] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:54.563776 7899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd348bcbe6594822a8fce339eca04905" candidate_uuid: "325b8f3915404001925a596ef97ed930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76634ab8eb914593903c21a6307a1a7e"
I20260812 03:57:54.563855 7899 raft_consensus.cc:3037] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:54.564494 7585 raft_consensus.cc:2454] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 325b8f3915404001925a596ef97ed930 in term 1.
I20260812 03:57:54.564705 7668 leader_election.cc:302] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325b8f3915404001925a596ef97ed930, bf97a7334d184009840330be17a07833; no voters:
I20260812 03:57:54.564797 7899 raft_consensus.cc:2454] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 325b8f3915404001925a596ef97ed930 in term 1.
I20260812 03:57:54.564847 7998 raft_consensus.cc:2781] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:54.565201 7998 raft_consensus.cc:686] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 1 LEADER]: Becoming Leader. State: Replica: 325b8f3915404001925a596ef97ed930, State: Running, Role: LEADER
I20260812 03:57:54.565340 7998 consensus_queue.cc:237] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:54.566305 6512 catalog_manager.cc:5495] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 reported cstate change: term changed from 0 to 1, leader changed from <none> to 325b8f3915404001925a596ef97ed930 (127.31.43.194). New cstate: current_term: 1 leader_uuid: "325b8f3915404001925a596ef97ed930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260812 03:57:54.573154 7986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.570510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41830 (local address 127.31.43.194:33159)
0812 03:57:54.570581 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.570583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.570591 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:57:54.570627 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.570629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.570655 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.570747 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:54.570918 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.571409 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.572197 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.572445 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.572528 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.572725 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:54.572848 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:54.572939 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.572979 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.573067 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:57:54.575271 7585 raft_consensus.cc:1261] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833 [term 1 FOLLOWER]: Refusing update from remote peer 325b8f3915404001925a596ef97ed930: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:54.575273 7899 raft_consensus.cc:1261] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e [term 1 FOLLOWER]: Refusing update from remote peer 325b8f3915404001925a596ef97ed930: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:54.575629 7998 consensus_queue.cc:1035] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:54.575821 7999 consensus_queue.cc:1035] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:54.578981 8002 mvcc.cc:205] Tried to move back new op lower bound from 7317532977456447488 to 7317532977420066816. Current Snapshot: MvccSnapshot[applied={T|T < 7317532977456447488}]
I20260812 03:57:54.580181 8003 mvcc.cc:205] Tried to move back new op lower bound from 7317532977456447488 to 7317532977420066816. Current Snapshot: MvccSnapshot[applied={T|T < 7317532977456447488}]
I20260812 03:57:54.581388 8005 mvcc.cc:205] Tried to move back new op lower bound from 7317532977456447488 to 7317532977420066816. Current Snapshot: MvccSnapshot[applied={T|T < 7317532977456447488}]
I20260812 03:57:54.582091 6512 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:54.582226 6512 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:54.583781 6512 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52dbb396108240fd9ce5b7b34010b505: Sending DeleteTablet for 3 replicas of tablet cd348bcbe6594822a8fce339eca04905
I20260812 03:57:54.584125 7717 tablet_service.cc:1514] Processing DeleteTablet for tablet cd348bcbe6594822a8fce339eca04905 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:54 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:41828
I20260812 03:57:54.584244 7565 tablet_service.cc:1514] Processing DeleteTablet for tablet cd348bcbe6594822a8fce339eca04905 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:54 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:45012
I20260812 03:57:54.584547 8014 tablet_replica.cc:330] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833: stopping tablet replica
I20260812 03:57:54.584630 7876 tablet_service.cc:1514] Processing DeleteTablet for tablet cd348bcbe6594822a8fce339eca04905 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:54 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:47020
I20260812 03:57:54.584702 8014 raft_consensus.cc:2229] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:54.584939 8014 raft_consensus.cc:2258] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:54.585155 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 6477
I20260812 03:57:54.588265 8017 tablet_replica.cc:330] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e: stopping tablet replica
I20260812 03:57:54.588419 8017 raft_consensus.cc:2229] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:54.588546 8017 raft_consensus.cc:2258] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:54.588974 8014 ts_tablet_manager.cc:1905] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:54.591328 8014 ts_tablet_manager.cc:1918] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:54.591425 8014 log.cc:1196] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-0/wal/wals/cd348bcbe6594822a8fce339eca04905
I20260812 03:57:54.591749 8014 ts_tablet_manager.cc:1939] T cd348bcbe6594822a8fce339eca04905 P bf97a7334d184009840330be17a07833: Deleting consensus metadata
I20260812 03:57:54.595937 8017 ts_tablet_manager.cc:1905] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:57:54.597564 7510 connection.cc:535] server connection from 127.0.0.1:45012 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:57:54.598541 7969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.585008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50554 (local address 127.31.43.252:34789)
0812 03:57:54.585060 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.585062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.588800 (+ 3738us) server_negotiation.cc:407] Connection header received
0812 03:57:54.588872 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.588873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.588892 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.588957 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.589841 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.590610 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.591654 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.592018 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.592101 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.593632 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.593633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.593634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.593642 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.595526 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.597239 (+ 1713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.597241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.597241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.597324 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.597570 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.597571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.597571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.597726 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.597814 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.598284 (+ 470us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.598410 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:57:54.599040 8017 ts_tablet_manager.cc:1918] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:54.599180 8017 log.cc:1196] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/cd348bcbe6594822a8fce339eca04905
I20260812 03:57:54.599551 8017 ts_tablet_manager.cc:1939] T cd348bcbe6594822a8fce339eca04905 P 76634ab8eb914593903c21a6307a1a7e: Deleting consensus metadata
W20260812 03:57:54.600005 7825 connection.cc:441] server connection from 127.0.0.1:47020 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47020 (request call id 1) could send its response
I20260812 03:57:54.602819 8013 tablet_replica.cc:330] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930: stopping tablet replica
I20260812 03:57:54.602982 8013 raft_consensus.cc:2229] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:57:54.603159 8013 raft_consensus.cc:2258] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:54.603809 8013 ts_tablet_manager.cc:1905] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:54.605746 8013 ts_tablet_manager.cc:1918] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:54.605839 8013 log.cc:1196] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/cd348bcbe6594822a8fce339eca04905
I20260812 03:57:54.606151 8013 ts_tablet_manager.cc:1939] T cd348bcbe6594822a8fce339eca04905 P 325b8f3915404001925a596ef97ed930: Deleting consensus metadata
W20260812 03:57:54.606613 7668 connection.cc:441] server connection from 127.0.0.1:41828 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41828 (request call id 1) could send its response
I20260812 03:57:54.610229 7966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.600613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.31.43.253:32979)
0812 03:57:54.600676 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.600678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.600686 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:57:54.600718 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.600719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.600731 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.600799 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.601556 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.602246 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.603824 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.604174 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.604218 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.606819 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.606822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.606822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.606832 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.608755 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.609178 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.609180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.609180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.609247 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.609488 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.609490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.609490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.609651 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.609730 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.609987 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.610118 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:57:54.610230 7969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:54.600918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50560 (local address 127.31.43.252:34789)
0812 03:57:54.600981 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:54.600982 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:54.601299 (+ 317us) server_negotiation.cc:407] Connection header received
0812 03:57:54.601343 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:54.601345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:54.601356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:54.601410 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:54.602589 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.603343 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.604369 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:54.604667 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:54.604712 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:54.607402 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:54.607403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:54.607403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:54.607412 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:54.608927 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.609387 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.609388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.609389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.609449 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:54.609670 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:54.609671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:54.609671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:54.609794 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:54.609913 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:54.610057 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 03:57:54.610139 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:57:55.581848 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:55.581526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.193:52297)
0812 03:57:55.581671 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:55.581781 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
W20260812 03:57:55.582096 7639 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:35427 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
I20260812 03:57:55.582474 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:55.582216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.194:40933)
0812 03:57:55.582346 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:55.582428 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260812 03:57:55.582551 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:55.582247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.195:34917)
0812 03:57:55.582412 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:55.582493 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":63,"threads_started":1}
W20260812 03:57:55.582744 7799 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:35427 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
W20260812 03:57:55.582809 7954 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:35427 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
I20260812 03:57:56.085029 8027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52dbb396108240fd9ce5b7b34010b505)
I20260812 03:57:56.085250 8027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } }
I20260812 03:57:56.085593 8027 leader_election.cc:288] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52dbb396108240fd9ce5b7b34010b505 (127.31.43.254:35427), 9a0da90f06dc41838c0eade213130e3a (127.31.43.252:34789)
I20260812 03:57:56.085995 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.085776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:58124)
0812 03:57:56.085896 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.085951 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
I20260812 03:57:56.085923 8029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52dbb396108240fd9ce5b7b34010b505)
W20260812 03:57:56.086112 7197 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
I20260812 03:57:56.086119 8029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } }
I20260812 03:57:56.086380 8029 leader_election.cc:288] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52dbb396108240fd9ce5b7b34010b505 (127.31.43.254:35427), 32f35d95da9e45b78a40bacdc04b6c53 (127.31.43.253:32979)
I20260812 03:57:56.086601 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.086482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:58128)
0812 03:57:56.086537 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.086571 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 03:57:56.086679 7197 leader_election.cc:334] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 52dbb396108240fd9ce5b7b34010b505 (127.31.43.254:35427): Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
I20260812 03:57:56.086804 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.086481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:58136)
0812 03:57:56.086579 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.086638 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":44,"threads_started":1}
W20260812 03:57:56.086946 7419 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
I20260812 03:57:56.087415 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.087312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:58148)
0812 03:57:56.087357 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.087390 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 03:57:56.087637 7419 leader_election.cc:334] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 52dbb396108240fd9ce5b7b34010b505 (127.31.43.254:35427): Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
I20260812 03:57:56.089265 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.085957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50570 (local address 127.31.43.252:34789)
0812 03:57:56.086066 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:56.086068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:56.086078 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:56.086332 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:56.086333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:56.086342 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:56.086416 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:56.086625 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.087430 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.088496 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.088918 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.089055 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.089070 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:56.089156 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 03:57:56.089204 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":56,"threads_started":1}
I20260812 03:57:56.089295 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.085797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34789 (local address 127.0.0.1:50570)
0812 03:57:56.085999 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.086011 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:56.086230 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:56.086475 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:56.086476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:56.086478 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:56.086542 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.086545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.087509 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.087510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.088396 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.088398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.089009 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.089010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.089071 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.089075 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:56.089132 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:57:56.089222 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":30,"thread_start_us":43,"threads_started":1}
I20260812 03:57:56.089679 7444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32f35d95da9e45b78a40bacdc04b6c53" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9a0da90f06dc41838c0eade213130e3a" is_pre_election: true
I20260812 03:57:56.089828 7444 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32f35d95da9e45b78a40bacdc04b6c53 in term 1.
I20260812 03:57:56.090001 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.086622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:32979 (local address 127.0.0.1:40222)
0812 03:57:56.086894 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.086904 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:56.086971 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:56.087395 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:56.087396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:56.087398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:56.087450 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.087452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.088219 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.088220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.088977 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.088979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.089741 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.089742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.089807 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.089811 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:56.089864 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 03:57:56.089911 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":79,"threads_started":1}
I20260812 03:57:56.090005 7194 leader_election.cc:302] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32f35d95da9e45b78a40bacdc04b6c53, 9a0da90f06dc41838c0eade213130e3a; no voters: 52dbb396108240fd9ce5b7b34010b505
I20260812 03:57:56.090167 8027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 03:57:56.090200 8027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52dbb396108240fd9ce5b7b34010b505)
I20260812 03:57:56.090224 8027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:57:56.090461 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.086747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40222 (local address 127.31.43.253:32979)
0812 03:57:56.087219 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:56.087221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:56.087230 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:56.087259 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:56.087259 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:56.087273 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:56.087343 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:56.087544 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.088121 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.089089 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.089620 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.089692 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.089703 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:56.090107 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 03:57:56.090395 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":447,"thread_start_us":82,"threads_started":1}
I20260812 03:57:56.090839 7224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a0da90f06dc41838c0eade213130e3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32f35d95da9e45b78a40bacdc04b6c53" is_pre_election: true
I20260812 03:57:56.091280 8027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } }
I20260812 03:57:56.091450 8027 leader_election.cc:288] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [CANDIDATE]: Term 2 election: Requested vote from peers 52dbb396108240fd9ce5b7b34010b505 (127.31.43.254:35427), 9a0da90f06dc41838c0eade213130e3a (127.31.43.252:34789)
I20260812 03:57:56.091629 7224 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a0da90f06dc41838c0eade213130e3a in current term 2: Already voted for candidate 32f35d95da9e45b78a40bacdc04b6c53 in this term.
I20260812 03:57:56.091657 7444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32f35d95da9e45b78a40bacdc04b6c53" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9a0da90f06dc41838c0eade213130e3a"
I20260812 03:57:56.091723 7444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:57:56.091845 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.091566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:58176)
0812 03:57:56.091786 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.091817 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204}
I20260812 03:57:56.092196 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.092094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:58178)
0812 03:57:56.092150 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.092174 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 03:57:56.092308 7197 leader_election.cc:334] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 52dbb396108240fd9ce5b7b34010b505 (127.31.43.254:35427): Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
I20260812 03:57:56.092545 7444 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32f35d95da9e45b78a40bacdc04b6c53 in term 2.
I20260812 03:57:56.092795 7194 leader_election.cc:302] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32f35d95da9e45b78a40bacdc04b6c53, 9a0da90f06dc41838c0eade213130e3a; no voters: 52dbb396108240fd9ce5b7b34010b505
I20260812 03:57:56.092829 7419 leader_election.cc:302] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a0da90f06dc41838c0eade213130e3a; no voters: 32f35d95da9e45b78a40bacdc04b6c53, 52dbb396108240fd9ce5b7b34010b505
I20260812 03:57:56.092897 8027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:57:56.093044 8027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [term 2 LEADER]: Becoming Leader. State: Replica: 32f35d95da9e45b78a40bacdc04b6c53, State: Running, Role: LEADER
I20260812 03:57:56.093036 8029 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:57:56.093119 8027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } }
I20260812 03:57:56.093497 8039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32f35d95da9e45b78a40bacdc04b6c53. Latest consensus state: current_term: 2 leader_uuid: "32f35d95da9e45b78a40bacdc04b6c53" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } } }
I20260812 03:57:56.093595 8039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:56.093772 8041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:56.094290 8041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:56.094362 8041 catalog_manager.cc:1259] Loaded cluster ID: 9eab5de793e84a0fb81fbd2348d322b7
I20260812 03:57:56.094379 8041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:56.094575 8041 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:56.094652 8041 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: Loaded TSK: 0
I20260812 03:57:56.094832 8041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:57:56.150569 7212 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:56.152385 7212 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:56.153617 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.153474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:58210)
0812 03:57:56.153540 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.153582 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 03:57:56.153648 7444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [term 2 FOLLOWER]: Refusing update from remote peer 32f35d95da9e45b78a40bacdc04b6c53: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 03:57:56.153954 8039 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 03:57:56.154125 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.153988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:58220)
0812 03:57:56.154044 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.154080 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 03:57:56.154256 7197 consensus_peers.cc:487] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 -> Peer 52dbb396108240fd9ce5b7b34010b505 (127.31.43.254:35427): Couldn't send request to peer 52dbb396108240fd9ce5b7b34010b505. Status: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 03:57:56.154711 8029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32f35d95da9e45b78a40bacdc04b6c53. Latest consensus state: current_term: 2 leader_uuid: "32f35d95da9e45b78a40bacdc04b6c53" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } } }
I20260812 03:57:56.154860 8029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:56.155207 8042 mvcc.cc:205] Tried to move back new op lower bound from 7317532983922630656 to 7317532983677947904. Current Snapshot: MvccSnapshot[applied={T|T < 7317532983922630656}]
I20260812 03:57:56.155303 8027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "32f35d95da9e45b78a40bacdc04b6c53" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } } }
I20260812 03:57:56.155422 8027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:56.155498 8044 mvcc.cc:205] Tried to move back new op lower bound from 7317532983922630656 to 7317532983677947904. Current Snapshot: MvccSnapshot[applied={T|T < 7317532983922630656}]
I20260812 03:57:56.155644 8029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "32f35d95da9e45b78a40bacdc04b6c53" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "52dbb396108240fd9ce5b7b34010b505" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35427 } } peers { permanent_uuid: "32f35d95da9e45b78a40bacdc04b6c53" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 32979 } attrs { promote: false } } peers { permanent_uuid: "9a0da90f06dc41838c0eade213130e3a" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34789 } attrs { promote: false } } }
I20260812 03:57:56.155741 8029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0da90f06dc41838c0eade213130e3a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:56.161113 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.157789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.31.43.194:33159)
0812 03:57:56.157903 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:56.157905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:56.157912 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:57:56.157937 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:56.157938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:56.157972 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:56.158212 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:56.158258 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.158966 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.160155 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.160763 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.160860 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.160874 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:56.160997 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 03:57:56.161044 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
I20260812 03:57:56.161110 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.157702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:33159 (local address 127.0.0.1:41838)
0812 03:57:56.157736 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.157749 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:56.157794 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:56.158070 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:56.158071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:56.158073 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:56.158133 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.158135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.159061 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.159062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.160049 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.160052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.160863 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.160864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.160926 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.160931 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:56.160983 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:57:56.161029 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":3}
I20260812 03:57:56.161517 7717 tablet_service.cc:1467] Processing CreateTablet for tablet 38296cb62b9444edaee57e583e14445e (DEFAULT_TABLE table=test-table [id=31dbca6e9c43439397c1aa63656321b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:56.161693 7717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38296cb62b9444edaee57e583e14445e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:56.161726 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.157861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:38991 (local address 127.0.0.1:47032)
0812 03:57:56.158143 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.158152 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:56.158196 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:56.158451 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:56.158452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:56.158455 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:56.158511 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.158512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.159510 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.159511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.160614 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.160616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.161484 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.161485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.161548 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.161553 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:56.161586 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 03:57:56.161642 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260812 03:57:56.161861 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.157776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:37083 (local address 127.0.0.1:45018)
0812 03:57:56.157873 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.157883 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:56.157934 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:56.158525 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:56.158526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:56.158529 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:56.158581 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.158582 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.159531 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.159532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.160491 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.160493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.161641 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.161642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.161707 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.161710 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:56.161738 (+ 28us) client_negotiation.cc:240] Negotiation successful
0812 03:57:56.161779 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":1}
I20260812 03:57:56.162109 8052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.158190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47032 (local address 127.31.43.195:38991)
0812 03:57:56.158318 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:56.158321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:56.158329 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:57:56.158354 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:56.158355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:56.158364 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:56.158427 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:56.158598 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.159413 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.160724 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.161376 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.161896 (+ 520us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.161958 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:56.161980 (+ 22us) server_negotiation.cc:299] Negotiation successful
0812 03:57:56.162040 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260812 03:57:56.162173 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.158038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45018 (local address 127.31.43.193:37083)
0812 03:57:56.158347 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:56.158349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:56.158357 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:57:56.158390 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:56.158391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:56.158400 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:56.158513 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:56.158661 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.159416 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.160581 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.161163 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.162015 (+ 852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.162030 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:56.162048 (+ 18us) server_negotiation.cc:299] Negotiation successful
0812 03:57:56.162092 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":62,"threads_started":1}
I20260812 03:57:56.162369 7565 tablet_service.cc:1467] Processing CreateTablet for tablet 38296cb62b9444edaee57e583e14445e (DEFAULT_TABLE table=test-table [id=31dbca6e9c43439397c1aa63656321b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:56.162389 7876 tablet_service.cc:1467] Processing CreateTablet for tablet 38296cb62b9444edaee57e583e14445e (DEFAULT_TABLE table=test-table [id=31dbca6e9c43439397c1aa63656321b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:56.162495 7876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38296cb62b9444edaee57e583e14445e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:56.162496 7565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38296cb62b9444edaee57e583e14445e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:56.163302 8053 tablet_bootstrap.cc:492] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930: Bootstrap starting.
I20260812 03:57:56.163763 8053 tablet_bootstrap.cc:654] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:56.164024 8055 tablet_bootstrap.cc:492] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e: Bootstrap starting.
I20260812 03:57:56.164455 8055 tablet_bootstrap.cc:654] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:56.164693 8053 tablet_bootstrap.cc:492] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930: No bootstrap required, opened a new log
I20260812 03:57:56.164758 8053 ts_tablet_manager.cc:1397] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:56.164801 8054 tablet_bootstrap.cc:492] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833: Bootstrap starting.
I20260812 03:57:56.164903 8053 raft_consensus.cc:348] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:56.165002 8053 raft_consensus.cc:374] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:56.165035 8053 raft_consensus.cc:729] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325b8f3915404001925a596ef97ed930, State: Initialized, Role: FOLLOWER
I20260812 03:57:56.165091 8053 consensus_queue.cc:260] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:56.165189 8055 tablet_bootstrap.cc:492] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e: No bootstrap required, opened a new log
I20260812 03:57:56.165234 8054 tablet_bootstrap.cc:654] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:56.165244 8055 ts_tablet_manager.cc:1397] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:56.165261 8053 ts_tablet_manager.cc:1428] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:57:56.165388 8055 raft_consensus.cc:348] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:56.165474 8055 raft_consensus.cc:374] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:56.165500 8055 raft_consensus.cc:729] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76634ab8eb914593903c21a6307a1a7e, State: Initialized, Role: FOLLOWER
I20260812 03:57:56.165582 8055 consensus_queue.cc:260] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:56.165803 8055 ts_tablet_manager.cc:1428] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:56.166144 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.165505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.194:45553)
0812 03:57:56.166063 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.166108 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":108,"threads_started":1}
W20260812 03:57:56.166271 7956 tablet.cc:2369] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:57:56.166462 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.166199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.195:37421)
0812 03:57:56.166333 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.166421 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":82,"threads_started":1}
I20260812 03:57:56.166778 8054 tablet_bootstrap.cc:492] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833: No bootstrap required, opened a new log
I20260812 03:57:56.166847 8054 ts_tablet_manager.cc:1397] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:56.166942 7800 heartbeater.cc:503] Master 127.31.43.253:32979 was elected leader, sending a full tablet report...
I20260812 03:57:56.166985 8054 raft_consensus.cc:348] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:56.167073 8054 raft_consensus.cc:374] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:56.167107 8054 raft_consensus.cc:729] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf97a7334d184009840330be17a07833, State: Initialized, Role: FOLLOWER
I20260812 03:57:56.167155 8054 consensus_queue.cc:260] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:56.167337 8054 ts_tablet_manager.cc:1428] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:56.167846 7955 heartbeater.cc:503] Master 127.31.43.253:32979 was elected leader, sending a full tablet report...
I20260812 03:57:56.167893 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.166907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.194:36531)
0812 03:57:56.167804 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.167867 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":873}
I20260812 03:57:56.167912 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.167588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.193:40103)
0812 03:57:56.167832 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.167870 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":85,"threads_started":1}
I20260812 03:57:56.168138 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.167611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.195:44725)
0812 03:57:56.168076 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.168104 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":107,"threads_started":1}
I20260812 03:57:56.168133 7640 heartbeater.cc:503] Master 127.31.43.253:32979 was elected leader, sending a full tablet report...
I20260812 03:57:56.168263 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.168176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.193:32957)
0812 03:57:56.168215 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.168237 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 03:57:56.348511 7642 tablet.cc:2369] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:57:56.436960 8062 raft_consensus.cc:482] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:56.437167 8062 raft_consensus.cc:504] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:56.437642 8062 leader_election.cc:288] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 325b8f3915404001925a596ef97ed930 (127.31.43.194:33159), bf97a7334d184009840330be17a07833 (127.31.43.193:37083)
I20260812 03:57:56.442224 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.437968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:58453 (local address 127.31.43.193:37083)
0812 03:57:56.438027 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:56.438029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:56.438106 (+ 77us) server_negotiation.cc:407] Connection header received
0812 03:57:56.438191 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:56.438193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:56.438206 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:56.438288 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:56.438576 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.439381 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.441113 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.441743 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.441825 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.441836 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:56.442065 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 03:57:56.442151 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:57:56.442243 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.437994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:44211 (local address 127.31.43.194:33159)
0812 03:57:56.438095 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:56.438097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:56.438130 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:57:56.438254 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:56.438255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:56.438266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:56.438323 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:56.438528 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.439412 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.441114 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.441732 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.441824 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.441837 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:56.442128 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 03:57:56.442181 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:57:56.442201 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.437927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:33159 (local address 127.31.43.195:44211)
0812 03:57:56.438074 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.438084 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:56.438142 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:56.438336 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:56.438336 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:56.438337 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:56.438436 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.438439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.439530 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.439532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.440978 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.440983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.441857 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.441858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.441950 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.441955 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:56.442099 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 03:57:56.442134 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":57}
I20260812 03:57:56.442209 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.437870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:37083 (local address 127.31.43.195:58453)
0812 03:57:56.438017 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.438030 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:56.438115 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:56.438326 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:56.438327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:56.438329 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:56.438436 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.438439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.439533 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.439534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.440978 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:56.440983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.441882 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.441883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:56.441950 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.441955 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:56.442022 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:57:56.442074 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":97}
I20260812 03:57:56.442829 7585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38296cb62b9444edaee57e583e14445e" candidate_uuid: "76634ab8eb914593903c21a6307a1a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf97a7334d184009840330be17a07833" is_pre_election: true
I20260812 03:57:56.442834 7742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38296cb62b9444edaee57e583e14445e" candidate_uuid: "76634ab8eb914593903c21a6307a1a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "325b8f3915404001925a596ef97ed930" is_pre_election: true
I20260812 03:57:56.442972 7585 raft_consensus.cc:2454] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76634ab8eb914593903c21a6307a1a7e in term 0.
I20260812 03:57:56.443014 7742 raft_consensus.cc:2454] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76634ab8eb914593903c21a6307a1a7e in term 0.
I20260812 03:57:56.443255 7824 leader_election.cc:302] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325b8f3915404001925a596ef97ed930, 76634ab8eb914593903c21a6307a1a7e; no voters:
I20260812 03:57:56.443492 8062 raft_consensus.cc:2781] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:56.443555 8062 raft_consensus.cc:482] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:56.443589 8062 raft_consensus.cc:3037] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:56.444604 8062 raft_consensus.cc:504] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:56.444836 8062 leader_election.cc:288] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [CANDIDATE]: Term 1 election: Requested vote from peers 325b8f3915404001925a596ef97ed930 (127.31.43.194:33159), bf97a7334d184009840330be17a07833 (127.31.43.193:37083)
I20260812 03:57:56.444993 7742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38296cb62b9444edaee57e583e14445e" candidate_uuid: "76634ab8eb914593903c21a6307a1a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "325b8f3915404001925a596ef97ed930"
I20260812 03:57:56.445087 7742 raft_consensus.cc:3037] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:56.445135 7585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38296cb62b9444edaee57e583e14445e" candidate_uuid: "76634ab8eb914593903c21a6307a1a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf97a7334d184009840330be17a07833"
I20260812 03:57:56.445230 7585 raft_consensus.cc:3037] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:56.445852 7742 raft_consensus.cc:2454] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76634ab8eb914593903c21a6307a1a7e in term 1.
I20260812 03:57:56.445874 7585 raft_consensus.cc:2454] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76634ab8eb914593903c21a6307a1a7e in term 1.
I20260812 03:57:56.446085 7827 leader_election.cc:302] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76634ab8eb914593903c21a6307a1a7e, bf97a7334d184009840330be17a07833; no voters:
I20260812 03:57:56.446195 8062 raft_consensus.cc:2781] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:56.446429 8062 raft_consensus.cc:686] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 1 LEADER]: Becoming Leader. State: Replica: 76634ab8eb914593903c21a6307a1a7e, State: Running, Role: LEADER
I20260812 03:57:56.446800 8062 consensus_queue.cc:237] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } }
I20260812 03:57:56.446843 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.446732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.195:52275)
0812 03:57:56.446776 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.446813 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 03:57:56.447692 7212 catalog_manager.cc:5495] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 76634ab8eb914593903c21a6307a1a7e (127.31.43.195). New cstate: current_term: 1 leader_uuid: "76634ab8eb914593903c21a6307a1a7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76634ab8eb914593903c21a6307a1a7e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38991 } health_report { overall_health: HEALTHY } } }
I20260812 03:57:56.520747 8052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.518105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47038 (local address 127.31.43.195:38991)
0812 03:57:56.518182 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:56.518184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:56.518197 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:57:56.518240 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:56.518241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:56.518261 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:56.518355 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:56.518511 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.519060 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.519871 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:56.520084 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:56.520169 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:56.520348 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:56.520469 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:56.520569 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:56.520611 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 03:57:56.520638 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:57:56.522298 7585 raft_consensus.cc:1261] T 38296cb62b9444edaee57e583e14445e P bf97a7334d184009840330be17a07833 [term 1 FOLLOWER]: Refusing update from remote peer 76634ab8eb914593903c21a6307a1a7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:56.522320 7742 raft_consensus.cc:1261] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930 [term 1 FOLLOWER]: Refusing update from remote peer 76634ab8eb914593903c21a6307a1a7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:56.522624 8062 consensus_queue.cc:1035] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf97a7334d184009840330be17a07833" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 37083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:56.522851 8068 consensus_queue.cc:1035] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "325b8f3915404001925a596ef97ed930" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 33159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:56.523761 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.523638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.194:53939)
0812 03:57:56.523688 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.523723 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 03:57:56.523931 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.523638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.193:51269)
0812 03:57:56.523868 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.523901 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204}
I20260812 03:57:56.525662 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.525423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.195:48719)
0812 03:57:56.525587 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.525623 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260812 03:57:56.526232 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.526111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.193:51975)
0812 03:57:56.526162 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.526196 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 03:57:56.526255 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.526143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.194:37337)
0812 03:57:56.526183 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.526216 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 03:57:56.526954 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.526867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.31.43.195:45127)
0812 03:57:56.526904 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.526928 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 03:57:56.528673 7210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:56.528793 7210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:56.530225 7210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32f35d95da9e45b78a40bacdc04b6c53: Sending DeleteTablet for 3 replicas of tablet 38296cb62b9444edaee57e583e14445e
I20260812 03:57:56.530664 7717 tablet_service.cc:1514] Processing DeleteTablet for tablet 38296cb62b9444edaee57e583e14445e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:56 UTC) from {username='oryx', principal='oryx/127.31.43.253@KRBTEST.COM'} at 127.0.0.1:41838
I20260812 03:57:56.530740 7565 tablet_service.cc:1514] Processing DeleteTablet for tablet 38296cb62b9444edaee57e583e14445e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:56 UTC) from {username='oryx', principal='oryx/127.31.43.253@KRBTEST.COM'} at 127.0.0.1:45018
I20260812 03:57:56.530999 7876 tablet_service.cc:1514] Processing DeleteTablet for tablet 38296cb62b9444edaee57e583e14445e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:56 UTC) from {username='oryx', principal='oryx/127.31.43.253@KRBTEST.COM'} at 127.0.0.1:47032
I20260812 03:57:56.531188 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 7388
I20260812 03:57:56.531224 8081 tablet_replica.cc:330] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930: stopping tablet replica
I20260812 03:57:56.531282 8081 raft_consensus.cc:2229] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:56.531339 8081 raft_consensus.cc:2258] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:56.533191 8081 ts_tablet_manager.cc:1905] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:56.535583 8081 ts_tablet_manager.cc:1918] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:56.535671 8081 log.cc:1196] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/38296cb62b9444edaee57e583e14445e
I20260812 03:57:56.535964 8081 ts_tablet_manager.cc:1939] T 38296cb62b9444edaee57e583e14445e P 325b8f3915404001925a596ef97ed930: Deleting consensus metadata
I20260812 03:57:56.536374 7193 catalog_manager.cc:4841] TS 325b8f3915404001925a596ef97ed930 (127.31.43.194:33159): tablet 38296cb62b9444edaee57e583e14445e (table test-table [id=31dbca6e9c43439397c1aa63656321b4]) successfully deleted
I20260812 03:57:56.541204 8084 tablet_replica.cc:330] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e: stopping tablet replica
I20260812 03:57:56.541307 8084 raft_consensus.cc:2229] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 1 LEADER]: Raft consensus shutting down.
W20260812 03:57:56.541363 8031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:56.541207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:37083 (local address 127.0.0.1:45032)
0812 03:57:56.541259 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.541296 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:37083: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 03:57:56.541425 8084 raft_consensus.cc:2258] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 03:57:56.541524 7197 catalog_manager.cc:4584] TS bf97a7334d184009840330be17a07833 (127.31.43.193:37083): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 38296cb62b9444edaee57e583e14445e: Network error: Client connection negotiation failed: client connection to 127.31.43.193:37083: connect: Connection reset by peer (error 104)
I20260812 03:57:56.541777 8084 ts_tablet_manager.cc:1905] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:56.541872 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 7648
I20260812 03:57:56.545392 8084 ts_tablet_manager.cc:1918] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:56.545487 8084 log.cc:1196] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/38296cb62b9444edaee57e583e14445e
I20260812 03:57:56.545821 8084 ts_tablet_manager.cc:1939] T 38296cb62b9444edaee57e583e14445e P 76634ab8eb914593903c21a6307a1a7e: Deleting consensus metadata
I20260812 03:57:56.547909 7194 catalog_manager.cc:4841] TS 76634ab8eb914593903c21a6307a1a7e (127.31.43.195:38991): tablet 38296cb62b9444edaee57e583e14445e (table test-table [id=31dbca6e9c43439397c1aa63656321b4]) successfully deleted
I20260812 03:57:56.548370 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.548230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:58222)
0812 03:57:56.548296 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.548338 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 03:57:56.548871 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:56.548760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35427 (local address 127.0.0.1:58234)
0812 03:57:56.548811 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:56.548837 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 03:57:56.556104 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 7806
I20260812 03:57:56.564685 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 7056
I20260812 03:57:56.574270 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 7274
2026-08-12T03:57:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5299 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:56 dist-test-slave-8tdl krb5kdc[8088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:56 dist-test-slave-8tdl krb5kdc[8088](info): set up 2 sockets
Aug 12 03:57:56 dist-test-slave-8tdl krb5kdc[8088](info): commencing operation
krb5kdc: starting...
W20260812 03:57:57.753321 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:57:57 dist-test-slave-8tdl krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.254.keytab.
Entry for principal oryx/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.254.keytab.
I20260812 03:57:57.899323 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:41437
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.254.keytab
--principal=oryx/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:41437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:57.987497 8104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:57.987782 8104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:57.987838 8104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:57.989158 8104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:57.989243 8104 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:57:57.989264 8104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:57.989286 8104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:57.989317 8104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:57.989341 8104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:57.991065 8104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:41437
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.254.keytab
--principal=oryx/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:41437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:57.991372 8104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:57.991637 8104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:57.996973 8111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:57.996986 8113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:57.997253 8109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:57.997596 8104 server_base.cc:1034] running on GCE node
I20260812 03:57:57.997884 8104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:57.998281 8104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:57.999442 8104 hybrid_clock.cc:648] HybridClock initialized: now 1786507077999405 us; error 56 us; skew 500 ppm
Aug 12 03:57:57 dist-test-slave-8tdl krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507077, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:58.002118 8104 init.cc:377] Logged in from keytab as oryx/127.31.43.254@KRBTEST.COM (short username oryx)
I20260812 03:57:58.002590 8104 webserver.cc:460] Webserver started at http://127.31.43.254:36085/ using document root <none> and password file <none>
I20260812 03:57:58.002890 8104 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:58.002942 8104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:58.003104 8104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:58.004245 8104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "056847fc3ec348158814208689a3e24c"
format_stamp: "Formatted at 2026-08-12 03:57:57 on dist-test-slave-8tdl"
server_key: "b69e556a036d8361810b027d41881088"
server_key_iv: "305b9df10393cec6e659c0bed32a2f82"
server_key_version: "encryptionkey@0"
I20260812 03:57:58.004642 8104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "056847fc3ec348158814208689a3e24c"
format_stamp: "Formatted at 2026-08-12 03:57:57 on dist-test-slave-8tdl"
server_key: "b69e556a036d8361810b027d41881088"
server_key_iv: "305b9df10393cec6e659c0bed32a2f82"
server_key_version: "encryptionkey@0"
I20260812 03:57:58.006350 8104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:58.007449 8120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:58.007687 8104 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:58.007805 8104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "056847fc3ec348158814208689a3e24c"
format_stamp: "Formatted at 2026-08-12 03:57:57 on dist-test-slave-8tdl"
server_key: "b69e556a036d8361810b027d41881088"
server_key_iv: "305b9df10393cec6e659c0bed32a2f82"
server_key_version: "encryptionkey@0"
I20260812 03:57:58.007906 8104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:58.018041 8104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:58.020641 8104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:58.020830 8104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:58.026577 8104 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:41437
I20260812 03:57:58.026625 8196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:41437 every 8 connection(s)
I20260812 03:57:58.027032 8104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:58.027858 8197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:58.030406 8197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c: Bootstrap starting.
I20260812 03:57:58.031136 8197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:58.031396 8197 log.cc:824] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:58.032249 8197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c: No bootstrap required, opened a new log
I20260812 03:57:58.034969 8197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056847fc3ec348158814208689a3e24c" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41437 } }
I20260812 03:57:58.035151 8197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:58.035183 8197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 056847fc3ec348158814208689a3e24c, State: Initialized, Role: FOLLOWER
I20260812 03:57:58.035355 8197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056847fc3ec348158814208689a3e24c" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41437 } }
I20260812 03:57:58.035454 8197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:58.035481 8197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:58.035521 8197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:58.035977 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 8104
I20260812 03:57:58.036093 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:57:58.036321 31919 external_mini_cluster.cc:1428] Setting key 9cb47f402947a94bab2128576ba23aa2
I20260812 03:57:58.036644 8197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056847fc3ec348158814208689a3e24c" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41437 } }
I20260812 03:57:58.036778 8197 leader_election.cc:302] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 056847fc3ec348158814208689a3e24c; no voters:
I20260812 03:57:58.037118 8197 leader_election.cc:288] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:58.037174 8201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:58.037290 8201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [term 1 LEADER]: Becoming Leader. State: Replica: 056847fc3ec348158814208689a3e24c, State: Running, Role: LEADER
I20260812 03:57:58.037370 8201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056847fc3ec348158814208689a3e24c" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41437 } }
I20260812 03:57:58.037423 8197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:58.037853 8201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "056847fc3ec348158814208689a3e24c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056847fc3ec348158814208689a3e24c" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41437 } } }
I20260812 03:57:58.037938 8201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:58.038003 8203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 056847fc3ec348158814208689a3e24c. Latest consensus state: current_term: 1 leader_uuid: "056847fc3ec348158814208689a3e24c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056847fc3ec348158814208689a3e24c" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41437 } } }
I20260812 03:57:58.038094 8203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:58.038429 8208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:58.039240 8208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:58.040922 8208 catalog_manager.cc:1347] Generated new cluster ID: eb435bc156b549079771de2f9914712f
I20260812 03:57:58.040989 8208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:57:58 dist-test-slave-8tdl krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:58.049324 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.037232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45668 (local address 127.31.43.254:41437)
0812 03:57:58.037489 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.037500 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.037537 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:57:58.037664 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.037669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.037716 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.037863 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:58.038734 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.040228 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.041283 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.041932 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.042065 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.045214 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:58.045221 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:58.045222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:58.045240 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:58.047467 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.047910 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.047914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.047915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.048051 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.048276 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.048276 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.048277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.048653 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:58.048838 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.048915 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.049073 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":81,"threads_started":1}
I20260812 03:57:58.056197 8208 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:58.056805 8208 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:58.058681 8208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c: Generated new TSK 0
I20260812 03:57:58.058902 8208 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab.
Entry for principal oryx/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab.
I20260812 03:57:58.098392 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:41437
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab
--principal=oryx/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:58.214277 8225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:58.214543 8225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:58.214586 8225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:58.215862 8225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:58.215940 8225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:58.215979 8225 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:57:58.216017 8225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:57:58.217615 8225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab
--principal=oryx/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.43.254:41437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:58.217913 8225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:58.218171 8225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:58.224531 8236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:58.224547 8233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:58.224582 8225 server_base.cc:1034] running on GCE node
W20260812 03:57:58.224576 8234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:58.225059 8225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:58.225323 8225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:58.226518 8225 hybrid_clock.cc:648] HybridClock initialized: now 1786507078226492 us; error 45 us; skew 500 ppm
Aug 12 03:57:58 dist-test-slave-8tdl krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507078, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:58.229408 8225 init.cc:377] Logged in from keytab as oryx/127.31.43.193@KRBTEST.COM (short username oryx)
I20260812 03:57:58.230016 8225 webserver.cc:460] Webserver started at http://127.31.43.193:43511/ using document root <none> and password file <none>
I20260812 03:57:58.230332 8225 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:58.230419 8225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:58.230599 8225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:58.231882 8225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "7dde6cf488674ea9b96ca973bf81ec4e"
format_stamp: "Formatted at 2026-08-12 03:57:58 on dist-test-slave-8tdl"
server_key: "906c20ad83fe1fe50ee066108a2d791b"
server_key_iv: "78576571a4284b2f81e344a11113c925"
server_key_version: "encryptionkey@0"
I20260812 03:57:58.232317 8225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "7dde6cf488674ea9b96ca973bf81ec4e"
format_stamp: "Formatted at 2026-08-12 03:57:58 on dist-test-slave-8tdl"
server_key: "906c20ad83fe1fe50ee066108a2d791b"
server_key_iv: "78576571a4284b2f81e344a11113c925"
server_key_version: "encryptionkey@0"
I20260812 03:57:58.233933 8225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:57:58.235168 8243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:58.235427 8225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:58.235613 8225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "7dde6cf488674ea9b96ca973bf81ec4e"
format_stamp: "Formatted at 2026-08-12 03:57:58 on dist-test-slave-8tdl"
server_key: "906c20ad83fe1fe50ee066108a2d791b"
server_key_iv: "78576571a4284b2f81e344a11113c925"
server_key_version: "encryptionkey@0"
I20260812 03:57:58.235705 8225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:58.261904 8225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:58.265023 8225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:58.265245 8225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:58.265887 8225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:58.266055 8225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:58.266125 8225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:58.266150 8225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:57:58 dist-test-slave-8tdl krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507078, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:58.274111 8225 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:38553
I20260812 03:57:58.274206 8376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:38553 every 8 connection(s)
I20260812 03:57:58.274603 8225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:57:58.276469 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 8225
I20260812 03:57:58.276647 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:57:58.276889 31919 external_mini_cluster.cc:1428] Setting key ba460a87a9d435cf24ca4c3aa0075331
I20260812 03:57:58.278103 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.266661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:40793 (local address 127.31.43.254:41437)
0812 03:57:58.266787 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.266789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.267037 (+ 248us) server_negotiation.cc:407] Connection header received
0812 03:57:58.267687 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.267689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.267703 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.267764 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:58.268751 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.269638 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.271132 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.271424 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.271523 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.274296 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:58.274298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:58.274299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:58.274308 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:58.276086 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.276676 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.276678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.276678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.276757 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.277150 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.277152 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.277152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.277351 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:58.277470 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.277881 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.278006 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 03:57:58.278435 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.266636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:41437 (local address 127.31.43.193:40793)
0812 03:57:58.266865 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:58.266906 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:58.267563 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:58.267852 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:58.267855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:58.268279 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:58.268627 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.268634 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.269785 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.269786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.271023 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.271027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.271562 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.271564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.271740 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.272386 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:58.272396 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:58.274178 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:58.276222 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:58.276230 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:58.276233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:58.276537 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:58.276842 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:58.276843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:58.276844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:58.277050 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:58.277461 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:58.277470 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:58.277668 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 03:57:58.277850 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 03:57:58.278185 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":120,"threads_started":1}
I20260812 03:57:58.279137 8378 heartbeater.cc:347] Connected to a master server at 127.31.43.254:41437
I20260812 03:57:58.279238 8378 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:58.279482 8378 heartbeater.cc:511] Master 127.31.43.254:41437 requested a full tablet report, sending...
I20260812 03:57:58.280407 8142 ts_manager.cc:194] Registered new tserver with Master: 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193:38553)
I20260812 03:57:58.281210 8142 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:40793
WARNING: no policy specified for oryx/127.31.43.194@KRBTEST.COM; defaulting to no policy
I20260812 03:57:58.290057 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.281730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:41437 (local address 127.31.43.193:41335)
0812 03:57:58.282837 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:58.282852 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:58.282946 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:58.283123 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:58.283124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:58.283402 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:58.283498 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.283501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.284211 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.284212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.285264 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.285267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.285663 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.285664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.285751 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.286352 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:58.286359 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:58.286867 (+ 508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:58.288732 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:58.288733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:58.288733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:58.288976 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:58.289303 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:58.289304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:58.289305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:58.289395 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:58.289736 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:58.289737 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:58.289784 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 03:57:58.289847 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:57:58.289951 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1068,"thread_start_us":101,"threads_started":1}
I20260812 03:57:58.290159 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.281809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:41335 (local address 127.31.43.254:41437)
0812 03:57:58.281867 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.281870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.282940 (+ 1070us) server_negotiation.cc:407] Connection header received
0812 03:57:58.283028 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.283030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.283039 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.283093 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:58.283588 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.284093 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.285380 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.285581 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.285636 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.286979 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:58.286981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:58.286981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:58.286990 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:58.288634 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.289138 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.289140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.289140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.289199 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.289511 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.289513 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.289513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.289659 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:58.289731 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.289941 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.290051 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab.
Entry for principal oryx/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab.
I20260812 03:57:58.311017 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:41437
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab
--principal=oryx/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:58.404739 8389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:58.404991 8389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:58.405037 8389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:58.406306 8389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:58.406385 8389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:58.406422 8389 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:57:58.406462 8389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:57:58.408227 8389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab
--principal=oryx/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.43.254:41437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:58.408535 8389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:58.408808 8389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:58.414597 8394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:58.414575 8397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:58.414575 8395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:58.415032 8389 server_base.cc:1034] running on GCE node
I20260812 03:57:58.415287 8389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:58.415588 8389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:58.416777 8389 hybrid_clock.cc:648] HybridClock initialized: now 1786507078416744 us; error 45 us; skew 500 ppm
Aug 12 03:57:58 dist-test-slave-8tdl krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507078, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:58.419734 8389 init.cc:377] Logged in from keytab as oryx/127.31.43.194@KRBTEST.COM (short username oryx)
I20260812 03:57:58.420339 8389 webserver.cc:460] Webserver started at http://127.31.43.194:36345/ using document root <none> and password file <none>
I20260812 03:57:58.420605 8389 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:58.420656 8389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:58.420828 8389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:58.422050 8389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "8209ef2322dd41ce83c845035129ac54"
format_stamp: "Formatted at 2026-08-12 03:57:58 on dist-test-slave-8tdl"
server_key: "b17965336b48ea165f9359341f6e2ea7"
server_key_iv: "5f81652aab9ab31011d3af6b994769b7"
server_key_version: "encryptionkey@0"
I20260812 03:57:58.422464 8389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "8209ef2322dd41ce83c845035129ac54"
format_stamp: "Formatted at 2026-08-12 03:57:58 on dist-test-slave-8tdl"
server_key: "b17965336b48ea165f9359341f6e2ea7"
server_key_iv: "5f81652aab9ab31011d3af6b994769b7"
server_key_version: "encryptionkey@0"
I20260812 03:57:58.424080 8389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:58.425011 8406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:58.425187 8389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:58.425300 8389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "8209ef2322dd41ce83c845035129ac54"
format_stamp: "Formatted at 2026-08-12 03:57:58 on dist-test-slave-8tdl"
server_key: "b17965336b48ea165f9359341f6e2ea7"
server_key_iv: "5f81652aab9ab31011d3af6b994769b7"
server_key_version: "encryptionkey@0"
I20260812 03:57:58.425390 8389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:58.462023 8389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:58.464526 8389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:58.464694 8389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:58.465361 8389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:58.465407 8389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:58.465432 8389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:58.465445 8389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:58.472788 8389 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:42585
I20260812 03:57:58.472903 8540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:42585 every 8 connection(s)
I20260812 03:57:58.473220 8389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:57:58 dist-test-slave-8tdl krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507078, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:58.477803 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.466073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:36825 (local address 127.31.43.254:41437)
0812 03:57:58.466150 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.466152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.466367 (+ 215us) server_negotiation.cc:407] Connection header received
0812 03:57:58.467114 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.467130 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.467144 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.467237 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:58.468186 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.469054 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.470890 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.471233 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.471318 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.474363 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:58.474365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:58.474365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:58.474374 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:58.476087 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.476609 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.476611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.476611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.476693 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.477079 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.477080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.477081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.477218 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:58.477287 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.477641 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.477732 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:57:58.477979 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.466002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:41437 (local address 127.31.43.194:36825)
0812 03:57:58.466220 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:58.466264 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:58.466981 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:58.467326 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:58.467331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:58.467742 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:58.468084 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.468091 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.469176 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.469177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.470759 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.470763 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.471369 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.471371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.471542 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.472273 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:58.472281 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:58.474231 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:58.476199 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:58.476212 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:58.476215 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:58.476500 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:58.476793 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:58.476793 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:58.476794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:58.476978 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:58.477319 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:58.477327 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:58.477484 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 03:57:58.477604 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 03:57:58.477774 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":103,"threads_started":1}
I20260812 03:57:58.478338 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 8389
I20260812 03:57:58.478482 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:57:58.478730 31919 external_mini_cluster.cc:1428] Setting key 9b534f194162c03c75b9731e3544048d
I20260812 03:57:58.478770 8541 heartbeater.cc:347] Connected to a master server at 127.31.43.254:41437
I20260812 03:57:58.478869 8541 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:58.479125 8541 heartbeater.cc:511] Master 127.31.43.254:41437 requested a full tablet report, sending...
I20260812 03:57:58.479687 8142 ts_manager.cc:194] Registered new tserver with Master: 8209ef2322dd41ce83c845035129ac54 (127.31.43.194:42585)
I20260812 03:57:58.480199 8142 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:36825
I20260812 03:57:58.488930 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.480198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:41437 (local address 127.31.43.194:37919)
0812 03:57:58.480365 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:58.480377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:58.480438 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:58.481843 (+ 1405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:58.481844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:58.482066 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:58.482123 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.482126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.483134 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.483135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.483676 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.483678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.484225 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.484226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.484287 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.484867 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:58.484875 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:58.485300 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:58.487554 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:58.487555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:58.487555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:58.487752 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:58.487983 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:58.487983 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:58.487984 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:58.488054 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:58.488450 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:58.488451 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:58.488540 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 03:57:58.488666 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 03:57:58.488817 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260812 03:57:58.488973 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.481580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:37919 (local address 127.31.43.254:41437)
0812 03:57:58.481669 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.481672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.481684 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:57:58.481713 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.481714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.481727 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.481809 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:58.482209 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.482982 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.483768 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.484105 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.484194 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.485406 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:58.485407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:58.485408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:58.485417 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:58.487435 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.487836 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.487838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.487838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.487908 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.488127 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.488128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.488129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.488370 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:58.488447 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.488721 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.488861 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for oryx/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab.
Entry for principal oryx/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab.
I20260812 03:57:58.510825 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:41437
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab
--principal=oryx/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:40955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:58.601670 8550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:58.601904 8550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:58.601931 8550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:58.603232 8550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:58.603286 8550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:58.603307 8550 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:57:58.603331 8550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:57:58.604964 8550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:40955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab
--principal=oryx/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.43.254:41437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:58.605240 8550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:58.605500 8550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:58.611054 8556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:58.611023 8559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:58.611065 8557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:58.611373 8550 server_base.cc:1034] running on GCE node
I20260812 03:57:58.611656 8550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:58.611874 8550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:58.613047 8550 hybrid_clock.cc:648] HybridClock initialized: now 1786507078613020 us; error 41 us; skew 500 ppm
Aug 12 03:57:58 dist-test-slave-8tdl krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507078, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:58.615991 8550 init.cc:377] Logged in from keytab as oryx/127.31.43.195@KRBTEST.COM (short username oryx)
I20260812 03:57:58.616505 8550 webserver.cc:460] Webserver started at http://127.31.43.195:46721/ using document root <none> and password file <none>
I20260812 03:57:58.616744 8550 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:58.616794 8550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:58.616943 8550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:58.618067 8550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "f74bb1e00abe40fd8522a99fc52f7d88"
format_stamp: "Formatted at 2026-08-12 03:57:58 on dist-test-slave-8tdl"
server_key: "440a833d8a0d9284c9958d33f6aee81c"
server_key_iv: "803493d95135452ad82284e798213b3f"
server_key_version: "encryptionkey@0"
I20260812 03:57:58.618455 8550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "f74bb1e00abe40fd8522a99fc52f7d88"
format_stamp: "Formatted at 2026-08-12 03:57:58 on dist-test-slave-8tdl"
server_key: "440a833d8a0d9284c9958d33f6aee81c"
server_key_iv: "803493d95135452ad82284e798213b3f"
server_key_version: "encryptionkey@0"
I20260812 03:57:58.619985 8550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:58.620990 8567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:58.621289 8550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:58.621416 8550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "f74bb1e00abe40fd8522a99fc52f7d88"
format_stamp: "Formatted at 2026-08-12 03:57:58 on dist-test-slave-8tdl"
server_key: "440a833d8a0d9284c9958d33f6aee81c"
server_key_iv: "803493d95135452ad82284e798213b3f"
server_key_version: "encryptionkey@0"
I20260812 03:57:58.621551 8550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:58.630956 8550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:58.633715 8550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:58.633915 8550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:58.634567 8550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:57:58.634636 8550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:58.634709 8550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:57:58.634742 8550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:57:58 dist-test-slave-8tdl krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507078, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:58.643707 8550 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:34801
I20260812 03:57:58.643801 8694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:34801 every 8 connection(s)
I20260812 03:57:58.644160 8550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:57:58.646320 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.635312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:54695 (local address 127.31.43.254:41437)
0812 03:57:58.635403 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.635407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.635562 (+ 155us) server_negotiation.cc:407] Connection header received
0812 03:57:58.636079 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.636080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.636098 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.636188 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:58.636908 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.637778 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.639862 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.640204 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.640287 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.643037 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:58.643038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:58.643039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:58.643047 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:58.644799 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.645266 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.645267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.645268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.645334 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.645606 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.645608 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.645608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.645748 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:58.645827 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.646121 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.646224 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:57:58.646544 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.635255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:41437 (local address 127.31.43.195:54695)
0812 03:57:58.635416 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:58.635458 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:58.635969 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:58.636290 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:58.636295 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:58.636596 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:58.636799 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.636804 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.637892 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.637894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.639742 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.639747 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.640356 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.640358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.640574 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.641303 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:58.641322 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:58.642915 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:58.644909 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:58.644916 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:58.644918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:58.645180 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:58.645410 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:58.645411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:58.645412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:58.645535 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:58.645817 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:58.645824 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:58.645980 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 03:57:58.646095 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:57:58.646286 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260812 03:57:58.647181 8696 heartbeater.cc:347] Connected to a master server at 127.31.43.254:41437
I20260812 03:57:58.647305 8696 heartbeater.cc:464] Registering TS with master...
I20260812 03:57:58.647549 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 8550
I20260812 03:57:58.647655 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:57:58.647866 31919 external_mini_cluster.cc:1428] Setting key 6e20a917a027b8aee3bfa719dc84c236
I20260812 03:57:58.648221 8696 heartbeater.cc:511] Master 127.31.43.254:41437 requested a full tablet report, sending...
I20260812 03:57:58.648607 8141 ts_manager.cc:194] Registered new tserver with Master: f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801)
I20260812 03:57:58.649053 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:57:58.649219 8141 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:54695
I20260812 03:57:58.655556 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.648441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:44969 (local address 127.31.43.254:41437)
0812 03:57:58.648490 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.648492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.648591 (+ 99us) server_negotiation.cc:407] Connection header received
0812 03:57:58.648757 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.648759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.648772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.648866 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:58.649284 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.649783 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.650919 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.651233 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.651277 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.652519 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:58.652521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:58.652521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:58.652528 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:58.654107 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.654586 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.654588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.654588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.654660 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.654965 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.654966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.654966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.655099 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:58.655183 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.655359 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.655448 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:57:58.655612 8702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.648239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:41437 (local address 127.31.43.195:44969)
0812 03:57:58.648524 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:58.648536 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:58.648604 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:58.648873 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:58.648874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:58.649123 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:58.649200 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.649203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.649893 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.649894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.650809 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.650813 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.651371 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.651372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.651458 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.652039 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:58.652045 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:58.652403 (+ 358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:58.654230 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:58.654232 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:58.654232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:58.654493 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:58.654787 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:58.654788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:58.654789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:58.654863 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:58.655175 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:58.655176 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:58.655227 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 03:57:58.655315 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 03:57:58.655493 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":73,"threads_started":1}
Aug 12 03:57:58 dist-test-slave-8tdl krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507078, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:57:58 dist-test-slave-8tdl krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:57:58 dist-test-slave-8tdl krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:57:58.675765 8205 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:58.669338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45676 (local address 127.31.43.254:41437)
0812 03:57:58.669408 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.669410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.669419 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:58.669447 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.669448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.669460 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.669524 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:58.670183 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.670786 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.671634 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.671829 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.671896 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.675667 (+ 3771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45676: BlockingRecv error: recv got EOF from 127.0.0.1:45676 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 12 03:57:58 dist-test-slave-8tdl krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507078, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:58.685277 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.676729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45678 (local address 127.31.43.254:41437)
0812 03:57:58.676785 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.676787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.676832 (+ 45us) server_negotiation.cc:407] Connection header received
0812 03:57:58.676911 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.676912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.676922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.676969 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:58.677640 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.678266 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.679204 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.679410 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.679483 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.682256 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:58.682258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:58.682259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:58.682275 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:58.683945 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.684364 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.684365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.684370 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.684445 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:58.684666 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:58.684667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:58.684667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:58.684794 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:58.684911 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.685069 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.685164 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:57:58.687374 8141 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:58.689297 8141 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:58.697031 8721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.692746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:38553 (local address 127.0.0.1:33002)
0812 03:57:58.693333 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:58.693342 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:58.693386 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:58.693673 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:58.693674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:58.693676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:58.693731 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.693733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.694605 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.694607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.695813 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.695816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.696683 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.696684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.696773 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.696789 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 03:57:58.696871 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:57:58.696938 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":84,"threads_started":1}
I20260812 03:57:58.697058 8727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.692890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33002 (local address 127.31.43.193:38553)
0812 03:57:58.693131 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.693136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.693458 (+ 322us) server_negotiation.cc:407] Connection header received
0812 03:57:58.693561 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.693563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.693579 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.693676 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:58.693790 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.694509 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.695911 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.696457 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.696553 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.696643 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.696911 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.696967 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":126,"threads_started":1}
I20260812 03:57:58.697674 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.692509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:34801 (local address 127.0.0.1:35444)
0812 03:57:58.693005 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:58.693141 (+ 136us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:58.693221 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:58.693510 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:58.693513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:58.693515 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:58.693626 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.693629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.695171 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.695172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.696438 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.696441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.697416 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.697417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.697500 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.697505 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:57:58.697562 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:57:58.697603 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":42,"thread_start_us":78,"threads_started":1}
I20260812 03:57:58.698069 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.692718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35444 (local address 127.31.43.195:34801)
0812 03:57:58.692992 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.692996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.693306 (+ 310us) server_negotiation.cc:407] Connection header received
0812 03:57:58.693349 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.693351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.693369 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.693436 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:58.693715 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.695044 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.696544 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.697325 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.697401 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.697507 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.697921 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.697970 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":105,"threads_started":1}
I20260812 03:57:58.698014 8300 tablet_service.cc:1467] Processing CreateTablet for tablet 722c70c5bf754c0598f1c54ac17ab0ed (DEFAULT_TABLE table=test-table [id=3194cd2189e7427eb11961f760592b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:58.698490 8300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 722c70c5bf754c0598f1c54ac17ab0ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:58.698647 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.692509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:42585 (local address 127.0.0.1:39822)
0812 03:57:58.693155 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:58.693166 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:58.693235 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:58.693510 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:58.693513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:58.693551 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:58.693626 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.693628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.695214 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.695215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.696437 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.696441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.698385 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.698387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.698468 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.698472 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:58.698535 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:57:58.698575 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":130,"thread_start_us":92,"threads_started":1}
I20260812 03:57:58.698758 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.692698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39822 (local address 127.31.43.194:42585)
0812 03:57:58.692929 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.692935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.693235 (+ 300us) server_negotiation.cc:407] Connection header received
0812 03:57:58.693322 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.693324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.693343 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.693420 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:58.693724 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.695128 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.697552 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.698194 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.698268 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.698340 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.698554 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.698598 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":105,"threads_started":1}
I20260812 03:57:58.698953 8622 tablet_service.cc:1467] Processing CreateTablet for tablet 722c70c5bf754c0598f1c54ac17ab0ed (DEFAULT_TABLE table=test-table [id=3194cd2189e7427eb11961f760592b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:58.699368 8622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 722c70c5bf754c0598f1c54ac17ab0ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:58.699402 8465 tablet_service.cc:1467] Processing CreateTablet for tablet 722c70c5bf754c0598f1c54ac17ab0ed (DEFAULT_TABLE table=test-table [id=3194cd2189e7427eb11961f760592b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:58.699738 8465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 722c70c5bf754c0598f1c54ac17ab0ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:58.702004 8728 tablet_bootstrap.cc:492] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e: Bootstrap starting.
I20260812 03:57:58.702629 8728 tablet_bootstrap.cc:654] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:58.702970 8728 log.cc:824] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:58.704097 8728 tablet_bootstrap.cc:492] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e: No bootstrap required, opened a new log
I20260812 03:57:58.704193 8728 ts_tablet_manager.cc:1397] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:58.704259 8731 tablet_bootstrap.cc:492] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54: Bootstrap starting.
I20260812 03:57:58.704872 8731 tablet_bootstrap.cc:654] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:58.705199 8731 log.cc:824] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:58.705827 8728 raft_consensus.cc:348] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.705963 8731 tablet_bootstrap.cc:492] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54: No bootstrap required, opened a new log
I20260812 03:57:58.706010 8728 raft_consensus.cc:374] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:58.706014 8731 ts_tablet_manager.cc:1397] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:58.706063 8728 raft_consensus.cc:729] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde6cf488674ea9b96ca973bf81ec4e, State: Initialized, Role: FOLLOWER
I20260812 03:57:58.706192 8728 consensus_queue.cc:260] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.706552 8728 ts_tablet_manager.cc:1428] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:57:58.706564 8378 heartbeater.cc:503] Master 127.31.43.254:41437 was elected leader, sending a full tablet report...
I20260812 03:57:58.706781 8730 tablet_bootstrap.cc:492] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88: Bootstrap starting.
I20260812 03:57:58.707569 8730 tablet_bootstrap.cc:654] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:58.707929 8730 log.cc:824] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:58.708077 8731 raft_consensus.cc:348] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.708287 8731 raft_consensus.cc:374] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:58.708341 8731 raft_consensus.cc:729] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8209ef2322dd41ce83c845035129ac54, State: Initialized, Role: FOLLOWER
I20260812 03:57:58.708492 8731 consensus_queue.cc:260] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.708804 8731 ts_tablet_manager.cc:1428] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 03:57:58.708855 8541 heartbeater.cc:503] Master 127.31.43.254:41437 was elected leader, sending a full tablet report...
I20260812 03:57:58.708933 8730 tablet_bootstrap.cc:492] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88: No bootstrap required, opened a new log
I20260812 03:57:58.709007 8730 ts_tablet_manager.cc:1397] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 03:57:58.710954 8730 raft_consensus.cc:348] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.711104 8730 raft_consensus.cc:374] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:58.711127 8730 raft_consensus.cc:729] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74bb1e00abe40fd8522a99fc52f7d88, State: Initialized, Role: FOLLOWER
I20260812 03:57:58.711273 8730 consensus_queue.cc:260] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.711581 8730 ts_tablet_manager.cc:1428] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 03:57:58.711601 8696 heartbeater.cc:503] Master 127.31.43.254:41437 was elected leader, sending a full tablet report...
W20260812 03:57:58.724066 8542 tablet.cc:2369] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:57:58.735025 8735 raft_consensus.cc:482] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:58.735217 8735 raft_consensus.cc:504] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.735667 8735 leader_election.cc:288] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193:38553), f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801)
I20260812 03:57:58.739699 8727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.735933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:53429 (local address 127.31.43.193:38553)
0812 03:57:58.735992 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.735995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.736072 (+ 77us) server_negotiation.cc:407] Connection header received
0812 03:57:58.736153 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.736154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.736166 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.736226 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:58.736449 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.737323 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.738717 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.739237 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.739330 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.739341 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.739551 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.739606 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:57:58.739686 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.735884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:38553 (local address 127.31.43.194:53429)
0812 03:57:58.735990 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:58.736005 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:58.736077 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:58.736254 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:58.736255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:58.736257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:58.736345 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.736347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.737408 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.737409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.738561 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.738565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.739330 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.739331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.739445 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.739449 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:58.739530 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:57:58.739588 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":99}
I20260812 03:57:58.739773 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.736056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:59031 (local address 127.31.43.195:34801)
0812 03:57:58.736126 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.736128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.736269 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:57:58.736307 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.736308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.736323 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.736370 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:58.736545 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.737252 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.738802 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.739416 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.739495 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.739520 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.739652 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.739693 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:57:58.739864 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.735941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:34801 (local address 127.31.43.194:59031)
0812 03:57:58.736165 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:58.736174 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:58.736259 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:58.736397 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:58.736398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:58.736400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:58.736469 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.736471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.737405 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.737406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.738639 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:58.738644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.739515 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.739516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:58.739577 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.739580 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:58.739633 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 03:57:58.739664 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":100,"thread_start_us":143,"threads_started":1}
I20260812 03:57:58.740182 8322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "722c70c5bf754c0598f1c54ac17ab0ed" candidate_uuid: "8209ef2322dd41ce83c845035129ac54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" is_pre_election: true
I20260812 03:57:58.740336 8322 raft_consensus.cc:2454] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8209ef2322dd41ce83c845035129ac54 in term 0.
I20260812 03:57:58.740602 8409 leader_election.cc:302] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde6cf488674ea9b96ca973bf81ec4e, 8209ef2322dd41ce83c845035129ac54; no voters:
I20260812 03:57:58.740635 8645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "722c70c5bf754c0598f1c54ac17ab0ed" candidate_uuid: "8209ef2322dd41ce83c845035129ac54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" is_pre_election: true
I20260812 03:57:58.740783 8645 raft_consensus.cc:2454] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8209ef2322dd41ce83c845035129ac54 in term 0.
I20260812 03:57:58.740792 8735 raft_consensus.cc:2781] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:58.740877 8735 raft_consensus.cc:482] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:58.740918 8735 raft_consensus.cc:3037] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:58.742069 8735 raft_consensus.cc:504] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.742241 8735 leader_election.cc:288] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [CANDIDATE]: Term 1 election: Requested vote from peers 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193:38553), f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801)
I20260812 03:57:58.742420 8645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "722c70c5bf754c0598f1c54ac17ab0ed" candidate_uuid: "8209ef2322dd41ce83c845035129ac54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74bb1e00abe40fd8522a99fc52f7d88"
I20260812 03:57:58.742448 8322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "722c70c5bf754c0598f1c54ac17ab0ed" candidate_uuid: "8209ef2322dd41ce83c845035129ac54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde6cf488674ea9b96ca973bf81ec4e"
I20260812 03:57:58.742549 8322 raft_consensus.cc:3037] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:58.742554 8645 raft_consensus.cc:3037] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:58.743527 8322 raft_consensus.cc:2454] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8209ef2322dd41ce83c845035129ac54 in term 1.
I20260812 03:57:58.743652 8645 raft_consensus.cc:2454] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8209ef2322dd41ce83c845035129ac54 in term 1.
I20260812 03:57:58.743764 8409 leader_election.cc:302] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde6cf488674ea9b96ca973bf81ec4e, 8209ef2322dd41ce83c845035129ac54; no voters:
I20260812 03:57:58.743875 8735 raft_consensus.cc:2781] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:58.744086 8735 raft_consensus.cc:686] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 1 LEADER]: Becoming Leader. State: Replica: 8209ef2322dd41ce83c845035129ac54, State: Running, Role: LEADER
I20260812 03:57:58.744202 8735 consensus_queue.cc:237] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.745182 8142 catalog_manager.cc:5495] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8209ef2322dd41ce83c845035129ac54 (127.31.43.194). New cstate: current_term: 1 leader_uuid: "8209ef2322dd41ce83c845035129ac54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } health_report { overall_health: UNKNOWN } } }
I20260812 03:57:58.760985 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:58.758329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39828 (local address 127.31.43.194:42585)
0812 03:57:58.758413 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:58.758415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:58.758424 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:58.758464 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:58.758465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:58.758487 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:58.758559 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:58.758816 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.759327 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.760169 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:58.760364 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:58.760422 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:58.760628 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:58.760737 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:58.760800 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:58.760838 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 03:57:58.760859 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:57:58.763216 8645 raft_consensus.cc:1261] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Refusing update from remote peer 8209ef2322dd41ce83c845035129ac54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:58.763244 8322 raft_consensus.cc:1261] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Refusing update from remote peer 8209ef2322dd41ce83c845035129ac54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:58.763621 8738 consensus_queue.cc:1035] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:58.763779 8735 consensus_queue.cc:1035] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:58.767397 8744 mvcc.cc:205] Tried to move back new op lower bound from 7317532994610528256 to 7317532994536845312. Current Snapshot: MvccSnapshot[applied={T|T < 7317532994610528256}]
I20260812 03:57:58.769042 8142 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:58.769169 8142 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:58.770139 8142 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c: Sending DeleteTablet for 3 replicas of tablet 722c70c5bf754c0598f1c54ac17ab0ed
I20260812 03:57:58.770511 8465 tablet_service.cc:1514] Processing DeleteTablet for tablet 722c70c5bf754c0598f1c54ac17ab0ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:58 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:39822
I20260812 03:57:58.770576 8300 tablet_service.cc:1514] Processing DeleteTablet for tablet 722c70c5bf754c0598f1c54ac17ab0ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:58 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:33002
I20260812 03:57:58.770578 8622 tablet_service.cc:1514] Processing DeleteTablet for tablet 722c70c5bf754c0598f1c54ac17ab0ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:58 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:35444
I20260812 03:57:58.770860 8754 tablet_replica.cc:330] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88: stopping tablet replica
I20260812 03:57:58.770874 8753 tablet_replica.cc:330] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54: stopping tablet replica
I20260812 03:57:58.770980 8753 raft_consensus.cc:2229] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:57:58.770980 8754 raft_consensus.cc:2229] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:58.771106 8754 raft_consensus.cc:2258] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:58.771111 8753 raft_consensus.cc:2258] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:58.771238 8142 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:58.771411 8142 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:58.771664 8754 ts_tablet_manager.cc:1905] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:58.772150 8753 ts_tablet_manager.cc:1905] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:58.772914 8622 tablet_service.cc:1467] Processing CreateTablet for tablet 55d4616a2e5c46989c7e1809fd762fc2 (DEFAULT_TABLE table=test-table [id=246b907ce224480e855c9516f493ab7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:58.773044 8622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d4616a2e5c46989c7e1809fd762fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:58.774073 8755 tablet_replica.cc:330] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e: stopping tablet replica
I20260812 03:57:58.774227 8755 raft_consensus.cc:2229] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:58.774407 8730 tablet_bootstrap.cc:492] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88: Bootstrap starting.
I20260812 03:57:58.774423 8755 raft_consensus.cc:2258] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:58.774578 8753 ts_tablet_manager.cc:1918] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:58.774662 8753 log.cc:1196] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/722c70c5bf754c0598f1c54ac17ab0ed
I20260812 03:57:58.774921 8730 tablet_bootstrap.cc:654] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:58.775028 8753 ts_tablet_manager.cc:1939] T 722c70c5bf754c0598f1c54ac17ab0ed P 8209ef2322dd41ce83c845035129ac54: Deleting consensus metadata
I20260812 03:57:58.774935 8465 tablet_service.cc:1467] Processing CreateTablet for tablet 55d4616a2e5c46989c7e1809fd762fc2 (DEFAULT_TABLE table=test-table [id=246b907ce224480e855c9516f493ab7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:58.775134 8465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d4616a2e5c46989c7e1809fd762fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:58.775147 8755 ts_tablet_manager.cc:1905] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:58.776273 8731 tablet_bootstrap.cc:492] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54: Bootstrap starting.
I20260812 03:57:58.776356 8300 tablet_service.cc:1467] Processing CreateTablet for tablet 55d4616a2e5c46989c7e1809fd762fc2 (DEFAULT_TABLE table=test-table [id=246b907ce224480e855c9516f493ab7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:58.776481 8300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d4616a2e5c46989c7e1809fd762fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:58.776744 8731 tablet_bootstrap.cc:654] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:58.777582 8755 ts_tablet_manager.cc:1918] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:58.777689 8755 log.cc:1196] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/wal/wals/722c70c5bf754c0598f1c54ac17ab0ed
I20260812 03:57:58.777730 8728 tablet_bootstrap.cc:492] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e: Bootstrap starting.
I20260812 03:57:58.778019 8755 ts_tablet_manager.cc:1939] T 722c70c5bf754c0598f1c54ac17ab0ed P 7dde6cf488674ea9b96ca973bf81ec4e: Deleting consensus metadata
I20260812 03:57:58.778231 8728 tablet_bootstrap.cc:654] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:58.778581 8122 catalog_manager.cc:4841] TS 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193:38553): tablet 722c70c5bf754c0598f1c54ac17ab0ed (table test-table [id=3194cd2189e7427eb11961f760592b13]) successfully deleted
I20260812 03:57:58.778728 8122 catalog_manager.cc:4841] TS 8209ef2322dd41ce83c845035129ac54 (127.31.43.194:42585): tablet 722c70c5bf754c0598f1c54ac17ab0ed (table test-table [id=3194cd2189e7427eb11961f760592b13]) successfully deleted
I20260812 03:57:58.778792 8754 ts_tablet_manager.cc:1918] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:58.778889 8754 log.cc:1196] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/722c70c5bf754c0598f1c54ac17ab0ed
I20260812 03:57:58.779227 8728 tablet_bootstrap.cc:492] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e: No bootstrap required, opened a new log
I20260812 03:57:58.779299 8728 ts_tablet_manager.cc:1397] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:58.779326 8754 ts_tablet_manager.cc:1939] T 722c70c5bf754c0598f1c54ac17ab0ed P f74bb1e00abe40fd8522a99fc52f7d88: Deleting consensus metadata
I20260812 03:57:58.779456 8728 raft_consensus.cc:348] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.779539 8728 raft_consensus.cc:374] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:58.779560 8728 raft_consensus.cc:729] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde6cf488674ea9b96ca973bf81ec4e, State: Initialized, Role: FOLLOWER
I20260812 03:57:58.779610 8728 consensus_queue.cc:260] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.779726 8728 ts_tablet_manager.cc:1428] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:58.779987 8124 catalog_manager.cc:4841] TS f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801): tablet 722c70c5bf754c0598f1c54ac17ab0ed (table test-table [id=3194cd2189e7427eb11961f760592b13]) successfully deleted
I20260812 03:57:58.780216 8731 tablet_bootstrap.cc:492] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54: No bootstrap required, opened a new log
I20260812 03:57:58.780297 8731 ts_tablet_manager.cc:1397] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260812 03:57:58.780471 8731 raft_consensus.cc:348] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.780555 8731 raft_consensus.cc:374] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:58.780581 8731 raft_consensus.cc:729] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8209ef2322dd41ce83c845035129ac54, State: Initialized, Role: FOLLOWER
I20260812 03:57:58.780637 8731 consensus_queue.cc:260] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.780730 8731 ts_tablet_manager.cc:1428] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 03:57:58.783234 8730 tablet_bootstrap.cc:492] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88: No bootstrap required, opened a new log
I20260812 03:57:58.783341 8730 ts_tablet_manager.cc:1397] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260812 03:57:58.783540 8730 raft_consensus.cc:348] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.783648 8730 raft_consensus.cc:374] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:58.783676 8730 raft_consensus.cc:729] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74bb1e00abe40fd8522a99fc52f7d88, State: Initialized, Role: FOLLOWER
I20260812 03:57:58.783741 8730 consensus_queue.cc:260] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:58.783881 8730 ts_tablet_manager.cc:1428] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260812 03:57:58.894912 8697 tablet.cc:2369] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:57:59.025756 8380 tablet.cc:2369] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:57:59.058910 8736 raft_consensus.cc:482] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:59.059058 8736 raft_consensus.cc:504] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.059449 8736 leader_election.cc:288] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193:38553), 8209ef2322dd41ce83c845035129ac54 (127.31.43.194:42585)
I20260812 03:57:59.062745 8727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.059810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:42573 (local address 127.31.43.193:38553)
0812 03:57:59.059871 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.059873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.059884 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:57:59.059921 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.059921 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.059932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.059998 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:59.060340 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.060838 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.061925 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.062309 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.062405 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.062416 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.062572 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.062620 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:57:59.062754 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.059665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:38553 (local address 127.31.43.195:42573)
0812 03:57:59.059742 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:59.059756 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:59.059829 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:59.060164 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:59.060165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:59.060166 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:59.060242 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.060243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.060937 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.060938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.061807 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.061810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.062405 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.062406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.062487 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.062490 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:59.062565 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:57:59.062619 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":71}
I20260812 03:57:59.062758 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.059886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:58779 (local address 127.31.43.194:42585)
0812 03:57:59.059936 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.059938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.060032 (+ 94us) server_negotiation.cc:407] Connection header received
0812 03:57:59.060059 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.060059 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.060071 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.060117 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:59.060322 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.060827 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.061922 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.062315 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.062409 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.062420 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.062606 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.062655 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:57:59.062870 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.059784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:42585 (local address 127.31.43.195:58779)
0812 03:57:59.059927 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:59.059937 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:59.059990 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:59.060137 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:59.060138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:59.060140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:59.060234 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.060235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.060928 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.060929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.061807 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.061810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.062430 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.062431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.062495 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.062498 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:59.062589 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 03:57:59.062638 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":5,"thread_start_us":70,"threads_started":1}
I20260812 03:57:59.063035 8322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d4616a2e5c46989c7e1809fd762fc2" candidate_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" is_pre_election: true
I20260812 03:57:59.063131 8322 raft_consensus.cc:2454] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f74bb1e00abe40fd8522a99fc52f7d88 in term 0.
I20260812 03:57:59.063145 8489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d4616a2e5c46989c7e1809fd762fc2" candidate_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8209ef2322dd41ce83c845035129ac54" is_pre_election: true
I20260812 03:57:59.063335 8489 raft_consensus.cc:2454] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f74bb1e00abe40fd8522a99fc52f7d88 in term 0.
I20260812 03:57:59.063493 8569 leader_election.cc:302] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde6cf488674ea9b96ca973bf81ec4e, f74bb1e00abe40fd8522a99fc52f7d88; no voters:
I20260812 03:57:59.063637 8736 raft_consensus.cc:2781] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:59.063673 8736 raft_consensus.cc:482] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:59.063696 8736 raft_consensus.cc:3037] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:59.064574 8736 raft_consensus.cc:504] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.064752 8736 leader_election.cc:288] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [CANDIDATE]: Term 1 election: Requested vote from peers 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193:38553), 8209ef2322dd41ce83c845035129ac54 (127.31.43.194:42585)
I20260812 03:57:59.064935 8322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d4616a2e5c46989c7e1809fd762fc2" candidate_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde6cf488674ea9b96ca973bf81ec4e"
I20260812 03:57:59.064949 8489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d4616a2e5c46989c7e1809fd762fc2" candidate_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8209ef2322dd41ce83c845035129ac54"
I20260812 03:57:59.065029 8322 raft_consensus.cc:3037] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:59.065052 8489 raft_consensus.cc:3037] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:59.065752 8322 raft_consensus.cc:2454] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f74bb1e00abe40fd8522a99fc52f7d88 in term 1.
I20260812 03:57:59.065757 8489 raft_consensus.cc:2454] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f74bb1e00abe40fd8522a99fc52f7d88 in term 1.
I20260812 03:57:59.065949 8569 leader_election.cc:302] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8209ef2322dd41ce83c845035129ac54, f74bb1e00abe40fd8522a99fc52f7d88; no voters:
I20260812 03:57:59.066068 8736 raft_consensus.cc:2781] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:59.066229 8736 raft_consensus.cc:686] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 LEADER]: Becoming Leader. State: Replica: f74bb1e00abe40fd8522a99fc52f7d88, State: Running, Role: LEADER
I20260812 03:57:59.066294 8736 consensus_queue.cc:237] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.067209 8142 catalog_manager.cc:5495] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 reported cstate change: term changed from 0 to 1, leader changed from <none> to f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195). New cstate: current_term: 1 leader_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } health_report { overall_health: HEALTHY } } }
Aug 12 03:57:59 dist-test-slave-8tdl krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507077, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.254@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:59.159976 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.151755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45688 (local address 127.31.43.254:41437)
0812 03:57:59.151835 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.151837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.152159 (+ 322us) server_negotiation.cc:407] Connection header received
0812 03:57:59.152193 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.152194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.152205 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.152259 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:59.152831 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.153360 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.154215 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.154410 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.154477 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.156943 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:59.156944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:59.156945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:59.156953 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:59.158554 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:59.159134 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:59.159135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:59.159135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:59.159199 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:59.159433 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:59.159434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:59.159434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:59.159554 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:59.159613 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.159814 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.159904 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:57:59.160235 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.151677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:41437 (local address 127.0.0.1:45688)
0812 03:57:59.151967 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:59.151980 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:59.152057 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:59.152310 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:59.152312 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:59.152675 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:59.152738 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.152740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.153455 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.153456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.154087 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.154090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.154515 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.154516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.154575 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.155201 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:59.155207 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:59.156845 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:59.158664 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:59.158743 (+ 79us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:59.158747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:59.159034 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:59.159293 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:59.159294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:59.159294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:59.159365 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:59.159648 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:59.159657 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:59.159711 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 03:57:59.159782 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:57:59.160081 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":97,"thread_start_us":138,"threads_started":1}
I20260812 03:57:59.162577 8142 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:45688:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:57:59.162896 8142 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:59.164729 8465 tablet_service.cc:1467] Processing CreateTablet for tablet e9e9271dd6ce4a889369054736649795 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ebda5ea953e1423f8977f8f03457ff26]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:57:59.164734 8300 tablet_service.cc:1467] Processing CreateTablet for tablet e9e9271dd6ce4a889369054736649795 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ebda5ea953e1423f8977f8f03457ff26]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:57:59.164891 8465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9e9271dd6ce4a889369054736649795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:59.164928 8300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9e9271dd6ce4a889369054736649795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:59.166137 8731 tablet_bootstrap.cc:492] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: Bootstrap starting.
I20260812 03:57:59.166347 8622 tablet_service.cc:1467] Processing CreateTablet for tablet e9e9271dd6ce4a889369054736649795 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ebda5ea953e1423f8977f8f03457ff26]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:57:59.166487 8622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9e9271dd6ce4a889369054736649795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:59.166532 8731 tablet_bootstrap.cc:654] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:59.167207 8731 tablet_bootstrap.cc:492] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: No bootstrap required, opened a new log
I20260812 03:57:59.167254 8731 ts_tablet_manager.cc:1397] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:59.167394 8731 raft_consensus.cc:348] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.167479 8731 raft_consensus.cc:374] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:59.167497 8731 raft_consensus.cc:729] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8209ef2322dd41ce83c845035129ac54, State: Initialized, Role: FOLLOWER
I20260812 03:57:59.167555 8731 consensus_queue.cc:260] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.167670 8731 ts_tablet_manager.cc:1428] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:59.167689 8738 tablet_replica.cc:439] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } } }
I20260812 03:57:59.167781 8738 tablet_replica.cc:442] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:59.168217 8730 tablet_bootstrap.cc:492] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88: Bootstrap starting.
I20260812 03:57:59.168273 8728 tablet_bootstrap.cc:492] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e: Bootstrap starting.
I20260812 03:57:59.168663 8730 tablet_bootstrap.cc:654] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:59.168732 8728 tablet_bootstrap.cc:654] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:59.169509 8730 tablet_bootstrap.cc:492] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88: No bootstrap required, opened a new log
I20260812 03:57:59.169559 8730 ts_tablet_manager.cc:1397] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:59.169718 8730 raft_consensus.cc:348] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.169793 8730 raft_consensus.cc:374] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:59.169808 8730 raft_consensus.cc:729] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74bb1e00abe40fd8522a99fc52f7d88, State: Initialized, Role: FOLLOWER
I20260812 03:57:59.169840 8730 consensus_queue.cc:260] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.169930 8730 ts_tablet_manager.cc:1428] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:59.169981 8736 tablet_replica.cc:439] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } } }
I20260812 03:57:59.170091 8736 tablet_replica.cc:442] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:59.170320 8728 tablet_bootstrap.cc:492] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e: No bootstrap required, opened a new log
I20260812 03:57:59.170373 8728 ts_tablet_manager.cc:1397] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:59.170573 8728 raft_consensus.cc:348] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.170707 8728 raft_consensus.cc:374] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:59.170737 8728 raft_consensus.cc:729] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde6cf488674ea9b96ca973bf81ec4e, State: Initialized, Role: FOLLOWER
I20260812 03:57:59.170809 8728 consensus_queue.cc:260] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.170924 8728 ts_tablet_manager.cc:1428] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:59.171061 8733 tablet_replica.cc:439] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } } }
I20260812 03:57:59.171141 8733 tablet_replica.cc:442] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:59.171692 8738 raft_consensus.cc:482] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:59.171785 8738 raft_consensus.cc:504] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.171941 8738 leader_election.cc:288] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193:38553), f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801)
I20260812 03:57:59.172101 8322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e9271dd6ce4a889369054736649795" candidate_uuid: "8209ef2322dd41ce83c845035129ac54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" is_pre_election: true
I20260812 03:57:59.172093 8645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e9271dd6ce4a889369054736649795" candidate_uuid: "8209ef2322dd41ce83c845035129ac54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" is_pre_election: true
I20260812 03:57:59.172183 8645 raft_consensus.cc:2454] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8209ef2322dd41ce83c845035129ac54 in term 0.
I20260812 03:57:59.172199 8322 raft_consensus.cc:2454] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8209ef2322dd41ce83c845035129ac54 in term 0.
I20260812 03:57:59.172329 8411 leader_election.cc:302] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7dde6cf488674ea9b96ca973bf81ec4e, 8209ef2322dd41ce83c845035129ac54, f74bb1e00abe40fd8522a99fc52f7d88; no voters:
I20260812 03:57:59.172430 8738 raft_consensus.cc:2781] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:59.172466 8738 raft_consensus.cc:482] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:59.172492 8738 raft_consensus.cc:3037] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:59.173204 8738 raft_consensus.cc:504] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.173367 8738 leader_election.cc:288] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [CANDIDATE]: Term 1 election: Requested vote from peers 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193:38553), f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801)
I20260812 03:57:59.173516 8322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e9271dd6ce4a889369054736649795" candidate_uuid: "8209ef2322dd41ce83c845035129ac54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde6cf488674ea9b96ca973bf81ec4e"
I20260812 03:57:59.173550 8645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e9271dd6ce4a889369054736649795" candidate_uuid: "8209ef2322dd41ce83c845035129ac54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74bb1e00abe40fd8522a99fc52f7d88"
I20260812 03:57:59.173609 8645 raft_consensus.cc:3037] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:59.173612 8322 raft_consensus.cc:3037] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:59.174221 8645 raft_consensus.cc:2454] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8209ef2322dd41ce83c845035129ac54 in term 1.
I20260812 03:57:59.174285 8322 raft_consensus.cc:2454] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8209ef2322dd41ce83c845035129ac54 in term 1.
I20260812 03:57:59.174450 8409 leader_election.cc:302] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde6cf488674ea9b96ca973bf81ec4e, 8209ef2322dd41ce83c845035129ac54; no voters:
I20260812 03:57:59.174569 8738 raft_consensus.cc:2781] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:59.174620 8738 raft_consensus.cc:686] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [term 1 LEADER]: Becoming Leader. State: Replica: 8209ef2322dd41ce83c845035129ac54, State: Running, Role: LEADER
I20260812 03:57:59.174666 8738 consensus_queue.cc:237] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.175000 8738 tablet_replica.cc:439] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: TxnStatusTablet state changed. Reason: New leader 8209ef2322dd41ce83c845035129ac54. Latest consensus state: current_term: 1 leader_uuid: "8209ef2322dd41ce83c845035129ac54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } } }
I20260812 03:57:59.175071 8738 tablet_replica.cc:442] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:57:59.175217 8774 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:57:59.175345 8774 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:57:59.175330 8142 catalog_manager.cc:5495] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8209ef2322dd41ce83c845035129ac54 (127.31.43.194). New cstate: current_term: 1 leader_uuid: "8209ef2322dd41ce83c845035129ac54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } health_report { overall_health: UNKNOWN } } }
I20260812 03:57:59.183702 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.180606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:42585 (local address 127.0.0.1:39840)
0812 03:57:59.180692 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:59.180701 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:59.180739 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:59.180892 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:59.180893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:59.180894 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:59.180949 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.180951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.181863 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.181927 (+ 64us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.182631 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.182634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.183236 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.183237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.183294 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.183315 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:59.183536 (+ 221us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:59.183546 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 03:57:59.183584 (+ 38us) client_negotiation.cc:240] Negotiation successful
0812 03:57:59.183635 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 03:57:59.183712 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.180667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.31.43.194:42585)
0812 03:57:59.180715 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.180717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.180729 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:57:59.180806 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.180807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.180818 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.180888 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:59.180990 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.181741 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.182823 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.183127 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.183206 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.183421 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:59.183461 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:59.183517 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.183599 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.183627 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:57:59.184963 8322 raft_consensus.cc:1261] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Refusing update from remote peer 8209ef2322dd41ce83c845035129ac54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:59.184960 8645 raft_consensus.cc:1261] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Refusing update from remote peer 8209ef2322dd41ce83c845035129ac54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:59.185248 8750 consensus_queue.cc:1035] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:59.185425 8738 consensus_queue.cc:1035] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:59.186100 8736 tablet_replica.cc:439] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88: TxnStatusTablet state changed. Reason: New leader 8209ef2322dd41ce83c845035129ac54. Latest consensus state: current_term: 1 leader_uuid: "8209ef2322dd41ce83c845035129ac54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } } }
I20260812 03:57:59.186214 8736 tablet_replica.cc:442] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:59.186197 8733 tablet_replica.cc:439] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e: TxnStatusTablet state changed. Reason: New leader 8209ef2322dd41ce83c845035129ac54. Latest consensus state: current_term: 1 leader_uuid: "8209ef2322dd41ce83c845035129ac54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } } }
I20260812 03:57:59.186321 8733 tablet_replica.cc:442] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:59.186950 8738 tablet_replica.cc:439] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8209ef2322dd41ce83c845035129ac54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } } }
I20260812 03:57:59.186980 8741 mvcc.cc:205] Tried to move back new op lower bound from 7317532996339466240 to 7317532996299821056. Current Snapshot: MvccSnapshot[applied={T|T < 7317532996339466240}]
I20260812 03:57:59.187045 8738 tablet_replica.cc:442] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:57:59.187091 8738 tablet_replica.cc:439] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8209ef2322dd41ce83c845035129ac54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } } }
I20260812 03:57:59.187134 8738 tablet_replica.cc:442] T e9e9271dd6ce4a889369054736649795 P 8209ef2322dd41ce83c845035129ac54: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:57:59.187289 8733 tablet_replica.cc:439] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8209ef2322dd41ce83c845035129ac54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } } }
I20260812 03:57:59.187381 8733 tablet_replica.cc:442] T e9e9271dd6ce4a889369054736649795 P 7dde6cf488674ea9b96ca973bf81ec4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:59.187670 8736 tablet_replica.cc:439] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8209ef2322dd41ce83c845035129ac54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } } }
I20260812 03:57:59.187789 8736 tablet_replica.cc:442] T e9e9271dd6ce4a889369054736649795 P f74bb1e00abe40fd8522a99fc52f7d88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:57:59.187685 8742 mvcc.cc:205] Tried to move back new op lower bound from 7317532996339466240 to 7317532996299821056. Current Snapshot: MvccSnapshot[applied={T|T < 7317532996339466240}]
I20260812 03:57:59.191928 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.189216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35454 (local address 127.31.43.195:34801)
0812 03:57:59.189258 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.189260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.189269 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:59.189311 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.189312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.189328 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.189400 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:59.189545 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.190050 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.191145 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.191338 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.191395 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.191573 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:59.191692 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:59.191749 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.191815 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.191834 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:57:59.198261 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.194917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:50519 (local address 127.31.43.194:42585)
0812 03:57:59.194988 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.194990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.195159 (+ 169us) server_negotiation.cc:407] Connection header received
0812 03:57:59.195230 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.195231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.195247 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.195295 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:59.195789 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.196351 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.197509 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.197694 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.197753 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.197982 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:59.198021 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:59.198051 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.198163 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.198188 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:57:59.198312 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.194802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:42585 (local address 127.31.43.195:50519)
0812 03:57:59.195076 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:59.195088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:59.195157 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:59.195299 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:59.195300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:59.195302 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:59.195687 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.195689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.196442 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.196443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.197419 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.197422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.197775 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.197776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.197851 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.197871 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:59.198092 (+ 221us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:59.198093 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:57:59.198139 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 03:57:59.198184 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":136,"threads_started":1}
I20260812 03:57:59.201195 8322 raft_consensus.cc:1261] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Refusing update from remote peer f74bb1e00abe40fd8522a99fc52f7d88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:59.201304 8489 raft_consensus.cc:1261] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Refusing update from remote peer f74bb1e00abe40fd8522a99fc52f7d88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:59.201490 8760 consensus_queue.cc:1035] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:59.201664 8760 consensus_queue.cc:1035] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:59.217954 8787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.207904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:34801 (local address 127.31.43.194:46215)
0812 03:57:59.214905 (+ 7001us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:59.214920 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:59.215044 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:59.215212 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:59.215213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:59.215214 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:59.215270 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.215271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.216197 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.216198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.217029 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.217033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.217532 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.217534 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.217591 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.217616 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:59.217799 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:59.217801 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:57:59.217837 (+ 36us) client_negotiation.cc:240] Negotiation successful
0812 03:57:59.217879 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6920,"thread_start_us":98,"threads_started":1}
I20260812 03:57:59.217957 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.208186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:46215 (local address 127.31.43.195:34801)
0812 03:57:59.208230 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.208232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.215008 (+ 6776us) server_negotiation.cc:407] Connection header received
0812 03:57:59.215114 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.215115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.215128 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.215204 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:59.215331 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.216108 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.217118 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.217438 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.217504 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.217703 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:59.217739 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:59.217793 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.217841 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.217865 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:57:59.226163 8140 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:59.226305 8140 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:59.227021 8140 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c: Sending DeleteTablet for 3 replicas of tablet 55d4616a2e5c46989c7e1809fd762fc2
I20260812 03:57:59.227344 8622 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d4616a2e5c46989c7e1809fd762fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:59 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:35444
I20260812 03:57:59.227451 8754 tablet_replica.cc:330] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88: stopping tablet replica
I20260812 03:57:59.227423 8300 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d4616a2e5c46989c7e1809fd762fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:59 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:33002
I20260812 03:57:59.227497 8754 raft_consensus.cc:2229] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:57:59.227517 8755 tablet_replica.cc:330] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e: stopping tablet replica
I20260812 03:57:59.227561 8755 raft_consensus.cc:2229] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:59.227589 8754 raft_consensus.cc:2258] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:59.227598 8755 raft_consensus.cc:2258] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:59.227960 8755 ts_tablet_manager.cc:1905] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:59.228096 8754 ts_tablet_manager.cc:1905] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:59.230005 8465 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d4616a2e5c46989c7e1809fd762fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:59 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:39822
I20260812 03:57:59.230129 8755 ts_tablet_manager.cc:1918] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:59.230197 8753 tablet_replica.cc:330] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54: stopping tablet replica
I20260812 03:57:59.230273 8753 raft_consensus.cc:2229] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:59.230294 8755 log.cc:1196] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/wal/wals/55d4616a2e5c46989c7e1809fd762fc2
I20260812 03:57:59.230314 8753 raft_consensus.cc:2258] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:59.230664 8755 ts_tablet_manager.cc:1939] T 55d4616a2e5c46989c7e1809fd762fc2 P 7dde6cf488674ea9b96ca973bf81ec4e: Deleting consensus metadata
I20260812 03:57:59.232633 8122 catalog_manager.cc:4841] TS 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193:38553): tablet 55d4616a2e5c46989c7e1809fd762fc2 (table test-table [id=246b907ce224480e855c9516f493ab7f]) successfully deleted
I20260812 03:57:59.232908 8754 ts_tablet_manager.cc:1918] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:59.232980 8754 log.cc:1196] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/55d4616a2e5c46989c7e1809fd762fc2
I20260812 03:57:59.233239 8754 ts_tablet_manager.cc:1939] T 55d4616a2e5c46989c7e1809fd762fc2 P f74bb1e00abe40fd8522a99fc52f7d88: Deleting consensus metadata
I20260812 03:57:59.233482 8753 ts_tablet_manager.cc:1905] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:59.233634 8124 catalog_manager.cc:4841] TS f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801): tablet 55d4616a2e5c46989c7e1809fd762fc2 (table test-table [id=246b907ce224480e855c9516f493ab7f]) successfully deleted
I20260812 03:57:59.235308 8753 ts_tablet_manager.cc:1918] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:59.235383 8753 log.cc:1196] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/55d4616a2e5c46989c7e1809fd762fc2
I20260812 03:57:59.235662 8753 ts_tablet_manager.cc:1939] T 55d4616a2e5c46989c7e1809fd762fc2 P 8209ef2322dd41ce83c845035129ac54: Deleting consensus metadata
I20260812 03:57:59.236253 8122 catalog_manager.cc:4841] TS 8209ef2322dd41ce83c845035129ac54 (127.31.43.194:42585): tablet 55d4616a2e5c46989c7e1809fd762fc2 (table test-table [id=246b907ce224480e855c9516f493ab7f]) successfully deleted
Aug 12 03:57:59 dist-test-slave-8tdl krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:57:59 dist-test-slave-8tdl krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:57:59 dist-test-slave-8tdl krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:57:59.272637 8205 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:59.259882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45698 (local address 127.31.43.254:41437)
0812 03:57:59.259944 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.259946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.260122 (+ 176us) server_negotiation.cc:407] Connection header received
0812 03:57:59.260191 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.260193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.260208 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.260267 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:59.261124 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.261959 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.263253 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.263569 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.263654 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.272563 (+ 8909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45698: BlockingRecv error: recv got EOF from 127.0.0.1:45698 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 12 03:57:59 dist-test-slave-8tdl krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM
I20260812 03:57:59.285856 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.273750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.31.43.254:41437)
0812 03:57:59.273797 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.273799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.273881 (+ 82us) server_negotiation.cc:407] Connection header received
0812 03:57:59.275392 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.275394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.275409 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.275444 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:59.276253 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.277169 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.278639 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.278994 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.279080 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.282314 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:59.282316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:59.282317 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:59.282326 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:59.284284 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:59.284768 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:59.284770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:59.284770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:59.284843 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:59.285088 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:59.285089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:59.285089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:59.285247 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:59.285354 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.285511 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.285610 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:57:59.287843 8140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:59.288061 8140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:59.289579 8622 tablet_service.cc:1467] Processing CreateTablet for tablet 279b022e8a444966b5a401aa7da8b039 (DEFAULT_TABLE table=test-table [id=33bc76c62f7b4914ad0cd36ca5b04473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:59.289631 8300 tablet_service.cc:1467] Processing CreateTablet for tablet 279b022e8a444966b5a401aa7da8b039 (DEFAULT_TABLE table=test-table [id=33bc76c62f7b4914ad0cd36ca5b04473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:59.289744 8622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 279b022e8a444966b5a401aa7da8b039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:59.289778 8300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 279b022e8a444966b5a401aa7da8b039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:59.291335 8730 tablet_bootstrap.cc:492] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88: Bootstrap starting.
I20260812 03:57:59.291431 8465 tablet_service.cc:1467] Processing CreateTablet for tablet 279b022e8a444966b5a401aa7da8b039 (DEFAULT_TABLE table=test-table [id=33bc76c62f7b4914ad0cd36ca5b04473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:59.291563 8465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 279b022e8a444966b5a401aa7da8b039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:59.291837 8730 tablet_bootstrap.cc:654] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:59.292991 8728 tablet_bootstrap.cc:492] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e: Bootstrap starting.
I20260812 03:57:59.293125 8730 tablet_bootstrap.cc:492] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88: No bootstrap required, opened a new log
I20260812 03:57:59.293205 8730 ts_tablet_manager.cc:1397] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:59.293465 8728 tablet_bootstrap.cc:654] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:59.293403 8730 raft_consensus.cc:348] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.293511 8730 raft_consensus.cc:374] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:59.293537 8730 raft_consensus.cc:729] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74bb1e00abe40fd8522a99fc52f7d88, State: Initialized, Role: FOLLOWER
I20260812 03:57:59.293587 8730 consensus_queue.cc:260] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.293680 8730 ts_tablet_manager.cc:1428] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:59.293826 8731 tablet_bootstrap.cc:492] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54: Bootstrap starting.
I20260812 03:57:59.294191 8731 tablet_bootstrap.cc:654] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:59.295615 8728 tablet_bootstrap.cc:492] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e: No bootstrap required, opened a new log
I20260812 03:57:59.295679 8728 ts_tablet_manager.cc:1397] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 03:57:59.295881 8731 tablet_bootstrap.cc:492] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54: No bootstrap required, opened a new log
I20260812 03:57:59.295842 8728 raft_consensus.cc:348] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.295928 8728 raft_consensus.cc:374] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:59.295938 8731 ts_tablet_manager.cc:1397] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:59.295953 8728 raft_consensus.cc:729] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde6cf488674ea9b96ca973bf81ec4e, State: Initialized, Role: FOLLOWER
I20260812 03:57:59.296005 8728 consensus_queue.cc:260] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.296103 8728 ts_tablet_manager.cc:1428] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:59.296101 8731 raft_consensus.cc:348] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.296176 8731 raft_consensus.cc:374] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:59.296200 8731 raft_consensus.cc:729] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8209ef2322dd41ce83c845035129ac54, State: Initialized, Role: FOLLOWER
I20260812 03:57:59.296245 8731 consensus_queue.cc:260] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.296348 8731 ts_tablet_manager.cc:1428] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:57:59.325519 8733 raft_consensus.cc:482] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:59.325656 8733 raft_consensus.cc:504] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.326093 8733 leader_election.cc:288] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8209ef2322dd41ce83c845035129ac54 (127.31.43.194:42585), f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801)
I20260812 03:57:59.330788 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.326545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:59701 (local address 127.31.43.194:42585)
0812 03:57:59.326591 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.326593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.326818 (+ 225us) server_negotiation.cc:407] Connection header received
0812 03:57:59.326926 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.326928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.326937 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.326966 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:59.327179 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.327969 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.329793 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.330349 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.330404 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.330414 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.330640 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.330707 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:57:59.330932 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.326547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:58971 (local address 127.31.43.195:34801)
0812 03:57:59.326594 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.326596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.326741 (+ 145us) server_negotiation.cc:407] Connection header received
0812 03:57:59.326866 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.326867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.326882 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.326916 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:57:59.327559 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.328344 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.329862 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.330457 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.330525 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.330533 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.330812 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.330856 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:57:59.331247 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.326324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:34801 (local address 127.31.43.193:58971)
0812 03:57:59.326647 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:59.326659 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:59.326776 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:59.326991 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:59.326991 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:59.326994 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:59.327429 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.327432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.328527 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.328529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.329740 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.329744 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.330611 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.330613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.330713 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.330717 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:57:59.331095 (+ 378us) client_negotiation.cc:240] Negotiation successful
0812 03:57:59.331149 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":109,"threads_started":1}
I20260812 03:57:59.331526 8645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279b022e8a444966b5a401aa7da8b039" candidate_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" is_pre_election: true
I20260812 03:57:59.331629 8645 raft_consensus.cc:2454] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dde6cf488674ea9b96ca973bf81ec4e in term 0.
I20260812 03:57:59.331840 8247 leader_election.cc:302] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde6cf488674ea9b96ca973bf81ec4e, f74bb1e00abe40fd8522a99fc52f7d88; no voters:
I20260812 03:57:59.332016 8733 raft_consensus.cc:2781] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:59.332024 8806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.326325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:42585 (local address 127.31.43.193:59701)
0812 03:57:59.326744 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:59.326757 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:59.326848 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:59.327028 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:59.327029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:59.327032 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:57:59.327108 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.327110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.328157 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.328158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.329684 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.329688 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.330476 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.330476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.330557 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.330560 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:59.331765 (+ 1205us) client_negotiation.cc:240] Negotiation successful
0812 03:57:59.331812 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":48,"thread_start_us":106,"threads_started":1}
I20260812 03:57:59.332077 8733 raft_consensus.cc:482] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:59.332337 8733 raft_consensus.cc:3037] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:59.332476 8489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279b022e8a444966b5a401aa7da8b039" candidate_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8209ef2322dd41ce83c845035129ac54" is_pre_election: true
I20260812 03:57:59.332562 8489 raft_consensus.cc:2454] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dde6cf488674ea9b96ca973bf81ec4e in term 0.
I20260812 03:57:59.333451 8733 raft_consensus.cc:504] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.333635 8733 leader_election.cc:288] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [CANDIDATE]: Term 1 election: Requested vote from peers 8209ef2322dd41ce83c845035129ac54 (127.31.43.194:42585), f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801)
I20260812 03:57:59.333819 8489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279b022e8a444966b5a401aa7da8b039" candidate_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8209ef2322dd41ce83c845035129ac54"
I20260812 03:57:59.333891 8489 raft_consensus.cc:3037] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:59.333853 8645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279b022e8a444966b5a401aa7da8b039" candidate_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74bb1e00abe40fd8522a99fc52f7d88"
I20260812 03:57:59.333935 8645 raft_consensus.cc:3037] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:59.334825 8489 raft_consensus.cc:2454] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde6cf488674ea9b96ca973bf81ec4e in term 1.
I20260812 03:57:59.334824 8645 raft_consensus.cc:2454] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde6cf488674ea9b96ca973bf81ec4e in term 1.
I20260812 03:57:59.335067 8247 leader_election.cc:302] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde6cf488674ea9b96ca973bf81ec4e, f74bb1e00abe40fd8522a99fc52f7d88; no voters:
I20260812 03:57:59.335170 8733 raft_consensus.cc:2781] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:59.335357 8733 raft_consensus.cc:686] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 LEADER]: Becoming Leader. State: Replica: 7dde6cf488674ea9b96ca973bf81ec4e, State: Running, Role: LEADER
I20260812 03:57:59.335450 8733 consensus_queue.cc:237] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.336304 8140 catalog_manager.cc:5495] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193). New cstate: current_term: 1 leader_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } health_report { overall_health: UNKNOWN } } }
I20260812 03:57:59.346544 8727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.343447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.31.43.193:38553)
0812 03:57:59.343500 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.343503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.343542 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:57:59.343582 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.343583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.343607 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.343668 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:59.343862 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.344365 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.345599 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.345820 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.345888 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.346138 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:59.346255 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:59.346361 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.346454 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.346479 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:57:59.347939 8489 raft_consensus.cc:1261] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Refusing update from remote peer 7dde6cf488674ea9b96ca973bf81ec4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:59.348031 8645 raft_consensus.cc:1261] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Refusing update from remote peer 7dde6cf488674ea9b96ca973bf81ec4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:59.348249 8807 consensus_queue.cc:1035] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:59.348398 8733 consensus_queue.cc:1035] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:59.352561 8140 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:59.352691 8140 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:59.353438 8140 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c: Sending DeleteTablet for 3 replicas of tablet 279b022e8a444966b5a401aa7da8b039
I20260812 03:57:59.353760 8465 tablet_service.cc:1514] Processing DeleteTablet for tablet 279b022e8a444966b5a401aa7da8b039 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:59 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:39822
I20260812 03:57:59.353778 8622 tablet_service.cc:1514] Processing DeleteTablet for tablet 279b022e8a444966b5a401aa7da8b039 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:59 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:35444
I20260812 03:57:59.353868 8753 tablet_replica.cc:330] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54: stopping tablet replica
I20260812 03:57:59.353886 8754 tablet_replica.cc:330] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88: stopping tablet replica
I20260812 03:57:59.353941 8754 raft_consensus.cc:2229] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:59.353942 8753 raft_consensus.cc:2229] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:59.353992 8753 raft_consensus.cc:2258] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:59.353995 8754 raft_consensus.cc:2258] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:59.354207 8300 tablet_service.cc:1514] Processing DeleteTablet for tablet 279b022e8a444966b5a401aa7da8b039 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:59 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:33002
I20260812 03:57:59.354295 8755 tablet_replica.cc:330] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e: stopping tablet replica
I20260812 03:57:59.354341 8755 raft_consensus.cc:2229] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:57:59.354444 8755 raft_consensus.cc:2258] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:59.354915 8755 ts_tablet_manager.cc:1905] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:59.354950 8140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:57:59.355119 8140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:57:59.356719 8755 ts_tablet_manager.cc:1918] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:59.356798 8755 log.cc:1196] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-0/wal/wals/279b022e8a444966b5a401aa7da8b039
I20260812 03:57:59.357024 8753 ts_tablet_manager.cc:1905] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:59.357100 8755 ts_tablet_manager.cc:1939] T 279b022e8a444966b5a401aa7da8b039 P 7dde6cf488674ea9b96ca973bf81ec4e: Deleting consensus metadata
I20260812 03:57:59.357316 8465 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9e2a6bfbb6453e8d1531361e685e6f (DEFAULT_TABLE table=test-table [id=a4501382dcb74f6ea3f7c20964ecd1c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:59.357437 8465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9e2a6bfbb6453e8d1531361e685e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:59.358541 8753 ts_tablet_manager.cc:1918] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:59.358597 8753 log.cc:1196] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/279b022e8a444966b5a401aa7da8b039
I20260812 03:57:59.358848 8753 ts_tablet_manager.cc:1939] T 279b022e8a444966b5a401aa7da8b039 P 8209ef2322dd41ce83c845035129ac54: Deleting consensus metadata
I20260812 03:57:59.359234 8754 ts_tablet_manager.cc:1905] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:59.359272 8731 tablet_bootstrap.cc:492] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54: Bootstrap starting.
I20260812 03:57:59.359654 8731 tablet_bootstrap.cc:654] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:59.360222 8300 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9e2a6bfbb6453e8d1531361e685e6f (DEFAULT_TABLE table=test-table [id=a4501382dcb74f6ea3f7c20964ecd1c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:59.360356 8300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9e2a6bfbb6453e8d1531361e685e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:59.360746 8754 ts_tablet_manager.cc:1918] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:57:59.360802 8754 log.cc:1196] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/279b022e8a444966b5a401aa7da8b039
I20260812 03:57:59.361035 8754 ts_tablet_manager.cc:1939] T 279b022e8a444966b5a401aa7da8b039 P f74bb1e00abe40fd8522a99fc52f7d88: Deleting consensus metadata
I20260812 03:57:59.361768 8728 tablet_bootstrap.cc:492] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e: Bootstrap starting.
I20260812 03:57:59.362258 8728 tablet_bootstrap.cc:654] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:59.362815 8124 catalog_manager.cc:4841] TS f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801): tablet 279b022e8a444966b5a401aa7da8b039 (table test-table [id=33bc76c62f7b4914ad0cd36ca5b04473]) successfully deleted
I20260812 03:57:59.362946 8122 catalog_manager.cc:4841] TS 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193:38553): tablet 279b022e8a444966b5a401aa7da8b039 (table test-table [id=33bc76c62f7b4914ad0cd36ca5b04473]) successfully deleted
I20260812 03:57:59.363006 8122 catalog_manager.cc:4841] TS 8209ef2322dd41ce83c845035129ac54 (127.31.43.194:42585): tablet 279b022e8a444966b5a401aa7da8b039 (table test-table [id=33bc76c62f7b4914ad0cd36ca5b04473]) successfully deleted
I20260812 03:57:59.363090 8728 tablet_bootstrap.cc:492] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e: No bootstrap required, opened a new log
I20260812 03:57:59.363142 8728 ts_tablet_manager.cc:1397] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:57:59.363216 8622 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9e2a6bfbb6453e8d1531361e685e6f (DEFAULT_TABLE table=test-table [id=a4501382dcb74f6ea3f7c20964ecd1c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:57:59.363338 8622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9e2a6bfbb6453e8d1531361e685e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:59.363299 8728 raft_consensus.cc:348] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.363373 8728 raft_consensus.cc:374] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:59.363394 8728 raft_consensus.cc:729] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde6cf488674ea9b96ca973bf81ec4e, State: Initialized, Role: FOLLOWER
I20260812 03:57:59.363430 8728 consensus_queue.cc:260] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.363534 8728 ts_tablet_manager.cc:1428] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 03:57:59.364794 8730 tablet_bootstrap.cc:492] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88: Bootstrap starting.
I20260812 03:57:59.365274 8730 tablet_bootstrap.cc:654] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:59.366734 8730 tablet_bootstrap.cc:492] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88: No bootstrap required, opened a new log
I20260812 03:57:59.366793 8730 ts_tablet_manager.cc:1397] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:57:59.366982 8730 raft_consensus.cc:348] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.367070 8730 raft_consensus.cc:374] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:59.367093 8730 raft_consensus.cc:729] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74bb1e00abe40fd8522a99fc52f7d88, State: Initialized, Role: FOLLOWER
I20260812 03:57:59.367144 8730 consensus_queue.cc:260] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.367242 8730 ts_tablet_manager.cc:1428] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:59.367993 8731 tablet_bootstrap.cc:492] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54: No bootstrap required, opened a new log
I20260812 03:57:59.368067 8731 ts_tablet_manager.cc:1397] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260812 03:57:59.368245 8731 raft_consensus.cc:348] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.368335 8731 raft_consensus.cc:374] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:59.368359 8731 raft_consensus.cc:729] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8209ef2322dd41ce83c845035129ac54, State: Initialized, Role: FOLLOWER
I20260812 03:57:59.368412 8731 consensus_queue.cc:260] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.368547 8731 ts_tablet_manager.cc:1428] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:59.466329 8733 raft_consensus.cc:482] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:57:59.466468 8733 raft_consensus.cc:504] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.466655 8733 leader_election.cc:288] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8209ef2322dd41ce83c845035129ac54 (127.31.43.194:42585), f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801)
I20260812 03:57:59.466953 8645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9e2a6bfbb6453e8d1531361e685e6f" candidate_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" is_pre_election: true
I20260812 03:57:59.466953 8489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9e2a6bfbb6453e8d1531361e685e6f" candidate_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8209ef2322dd41ce83c845035129ac54" is_pre_election: true
I20260812 03:57:59.467077 8489 raft_consensus.cc:2454] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dde6cf488674ea9b96ca973bf81ec4e in term 0.
I20260812 03:57:59.467077 8645 raft_consensus.cc:2454] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dde6cf488674ea9b96ca973bf81ec4e in term 0.
I20260812 03:57:59.467243 8247 leader_election.cc:302] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde6cf488674ea9b96ca973bf81ec4e, f74bb1e00abe40fd8522a99fc52f7d88; no voters:
I20260812 03:57:59.467360 8733 raft_consensus.cc:2781] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:57:59.467417 8733 raft_consensus.cc:482] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:57:59.467438 8733 raft_consensus.cc:3037] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:59.468493 8733 raft_consensus.cc:504] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.468663 8733 leader_election.cc:288] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [CANDIDATE]: Term 1 election: Requested vote from peers 8209ef2322dd41ce83c845035129ac54 (127.31.43.194:42585), f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801)
I20260812 03:57:59.468896 8645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9e2a6bfbb6453e8d1531361e685e6f" candidate_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74bb1e00abe40fd8522a99fc52f7d88"
I20260812 03:57:59.468984 8645 raft_consensus.cc:3037] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:59.468870 8489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9e2a6bfbb6453e8d1531361e685e6f" candidate_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8209ef2322dd41ce83c845035129ac54"
I20260812 03:57:59.469063 8489 raft_consensus.cc:3037] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:59.469776 8645 raft_consensus.cc:2454] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde6cf488674ea9b96ca973bf81ec4e in term 1.
I20260812 03:57:59.469820 8489 raft_consensus.cc:2454] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde6cf488674ea9b96ca973bf81ec4e in term 1.
I20260812 03:57:59.469974 8247 leader_election.cc:302] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde6cf488674ea9b96ca973bf81ec4e, f74bb1e00abe40fd8522a99fc52f7d88; no voters:
I20260812 03:57:59.470084 8733 raft_consensus.cc:2781] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:59.470162 8733 raft_consensus.cc:686] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 LEADER]: Becoming Leader. State: Replica: 7dde6cf488674ea9b96ca973bf81ec4e, State: Running, Role: LEADER
I20260812 03:57:59.470209 8733 consensus_queue.cc:237] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } }
I20260812 03:57:59.470960 8142 catalog_manager.cc:5495] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193). New cstate: current_term: 1 leader_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dde6cf488674ea9b96ca973bf81ec4e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 } health_report { overall_health: UNKNOWN } } }
I20260812 03:57:59.496400 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.493329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:55947 (local address 127.31.43.194:42585)
0812 03:57:59.493443 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.493445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.493561 (+ 116us) server_negotiation.cc:407] Connection header received
0812 03:57:59.493646 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.493647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.493660 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.493728 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:59.493882 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.494405 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.495549 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.495907 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.496001 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.496186 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:59.496223 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:59.496275 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.496317 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.496338 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 03:57:59.496433 8823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.493223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:42585 (local address 127.31.43.193:55947)
0812 03:57:59.493472 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:59.493484 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:59.493578 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:59.493733 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:59.493733 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:59.493736 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:59.493799 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.493801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.494496 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.494497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.495433 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.495437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.496002 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.496003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.496064 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.496086 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:59.496281 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:59.496283 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:57:59.496315 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 03:57:59.496365 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":92,"threads_started":1}
I20260812 03:57:59.498868 8489 raft_consensus.cc:1261] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Refusing update from remote peer 7dde6cf488674ea9b96ca973bf81ec4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:59.499050 8645 raft_consensus.cc:1261] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Refusing update from remote peer 7dde6cf488674ea9b96ca973bf81ec4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:59.499125 8807 consensus_queue.cc:1035] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8209ef2322dd41ce83c845035129ac54" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 42585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:59.499277 8733 consensus_queue.cc:1035] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f74bb1e00abe40fd8522a99fc52f7d88" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 34801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:59.506345 8727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.503917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:53821 (local address 127.31.43.193:38553)
0812 03:57:59.503965 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:59.503967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:59.503976 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:59.504040 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:59.504041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:59.504052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:59.504105 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:59.504252 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.504717 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.505472 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.505777 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.505885 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.506083 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:59.506122 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:59.506177 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:59.506243 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 03:57:59.506268 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:57:59.506371 8787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.503872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:38553 (local address 127.31.43.194:53821)
0812 03:57:59.503925 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:59.503939 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:59.503986 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:59.504131 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:59.504132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:59.504134 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:57:59.504187 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.504188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.504782 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.504783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.505354 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:59.505356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:59.505888 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:59.505889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:59.505955 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:59.505962 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:57:59.506191 (+ 229us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:57:59.506194 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:57:59.506240 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 03:57:59.506298 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 03:57:59.512620 8142 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:59.512732 8142 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:57:59.513401 8142 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 056847fc3ec348158814208689a3e24c: Sending DeleteTablet for 3 replicas of tablet 9b9e2a6bfbb6453e8d1531361e685e6f
I20260812 03:57:59.513687 8622 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9e2a6bfbb6453e8d1531361e685e6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:59 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:35444
I20260812 03:57:59.513687 8300 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9e2a6bfbb6453e8d1531361e685e6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:59 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:33002
I20260812 03:57:59.513792 8754 tablet_replica.cc:330] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88: stopping tablet replica
I20260812 03:57:59.513792 8755 tablet_replica.cc:330] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e: stopping tablet replica
I20260812 03:57:59.513839 8754 raft_consensus.cc:2229] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:59.513840 8755 raft_consensus.cc:2229] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:57:59.513875 8754 raft_consensus.cc:2258] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:59.513907 8755 raft_consensus.cc:2258] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:59.514102 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 8225
I20260812 03:57:59.514118 8465 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9e2a6bfbb6453e8d1531361e685e6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:57:59 UTC) from {username='oryx', principal='oryx/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:39822
I20260812 03:57:59.514211 8753 tablet_replica.cc:330] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54: stopping tablet replica
I20260812 03:57:59.514273 8753 raft_consensus.cc:2229] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:57:59.514305 8753 raft_consensus.cc:2258] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:57:59.514320 8755 ts_tablet_manager.cc:1905] T 9b9e2a6bfbb6453e8d1531361e685e6f P 7dde6cf488674ea9b96ca973bf81ec4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:59.514972 8753 ts_tablet_manager.cc:1905] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:59.516731 8753 ts_tablet_manager.cc:1918] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:59.516788 8753 log.cc:1196] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/9b9e2a6bfbb6453e8d1531361e685e6f
I20260812 03:57:59.517019 8753 ts_tablet_manager.cc:1939] T 9b9e2a6bfbb6453e8d1531361e685e6f P 8209ef2322dd41ce83c845035129ac54: Deleting consensus metadata
I20260812 03:57:59.517418 8754 ts_tablet_manager.cc:1905] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:57:59.517635 8122 catalog_manager.cc:4841] TS 8209ef2322dd41ce83c845035129ac54 (127.31.43.194:42585): tablet 9b9e2a6bfbb6453e8d1531361e685e6f (table test-table [id=a4501382dcb74f6ea3f7c20964ecd1c6]) successfully deleted
I20260812 03:57:59.519094 8754 ts_tablet_manager.cc:1918] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:57:59.519173 8754 log.cc:1196] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/9b9e2a6bfbb6453e8d1531361e685e6f
I20260812 03:57:59.519424 8754 ts_tablet_manager.cc:1939] T 9b9e2a6bfbb6453e8d1531361e685e6f P f74bb1e00abe40fd8522a99fc52f7d88: Deleting consensus metadata
I20260812 03:57:59.519948 8124 catalog_manager.cc:4841] TS f74bb1e00abe40fd8522a99fc52f7d88 (127.31.43.195:34801): tablet 9b9e2a6bfbb6453e8d1531361e685e6f (table test-table [id=a4501382dcb74f6ea3f7c20964ecd1c6]) successfully deleted
W20260812 03:57:59.526861 8122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:38553 (error 108)
I20260812 03:57:59.527215 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 8389
I20260812 03:57:59.529824 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:59.527334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:38553 (local address 127.0.0.1:33022)
0812 03:57:59.529702 (+ 2368us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:59.529765 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:38553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2344,"thread_start_us":85,"threads_started":1}
W20260812 03:57:59.533766 8122 catalog_manager.cc:4584] TS 7dde6cf488674ea9b96ca973bf81ec4e (127.31.43.193:38553): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9b9e2a6bfbb6453e8d1531361e685e6f: Network error: Client connection negotiation failed: client connection to 127.31.43.193:38553: connect: Connection refused (error 111)
I20260812 03:57:59.540767 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 8550
I20260812 03:57:59.551728 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 8104
2026-08-12T03:57:59Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2976 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:59 dist-test-slave-8tdl krb5kdc[8833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:59 dist-test-slave-8tdl krb5kdc[8833](info): set up 2 sockets
Aug 12 03:57:59 dist-test-slave-8tdl krb5kdc[8833](info): commencing operation
krb5kdc: starting...
W20260812 03:58:00.733273 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:58:00 dist-test-slave-8tdl krb5kdc[8833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:58:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:58:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:58:00.875337 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:36659
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:36659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:58:00.963640 8847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:00.963927 8847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:00.963968 8847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:00.965272 8847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:58:00.965349 8847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:00.965373 8847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:00.965408 8847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:58:00.965453 8847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:58:00.967198 8847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:36659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:36659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:00.967509 8847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:00.967772 8847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:00.973520 8853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:00.973464 8854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:00.973464 8856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:00.973742 8847 server_base.cc:1034] running on GCE node
I20260812 03:58:00.974123 8847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:00.974502 8847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:00.975695 8847 hybrid_clock.cc:648] HybridClock initialized: now 1786507080975662 us; error 50 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T03:58:00Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1429 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:58:01 dist-test-slave-8tdl krb5kdc[8861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:58:01 dist-test-slave-8tdl krb5kdc[8861](info): set up 2 sockets
Aug 12 03:58:01 dist-test-slave-8tdl krb5kdc[8861](info): commencing operation
krb5kdc: starting...
W20260812 03:58:02.121171 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:58:02 dist-test-slave-8tdl krb5kdc[8861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:58:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:58:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:58:02.267168 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:42917
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:42917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:58:02.365504 8875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:02.365793 8875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:02.365844 8875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:02.367269 8875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:58:02.367357 8875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:02.367386 8875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:02.367411 8875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:58:02.367445 8875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:58:02.369302 8875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:42917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:42917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:02.369637 8875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:02.369900 8875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:02.375367 8881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:02.375464 8885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:02.375367 8882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:02.375645 8875 server_base.cc:1034] running on GCE node
I20260812 03:58:02.375890 8875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:02.376233 8875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:02.377391 8875 hybrid_clock.cc:648] HybridClock initialized: now 1786507082377369 us; error 46 us; skew 500 ppm
Aug 12 03:58:02 dist-test-slave-8tdl krb5kdc[8861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507082, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:58:02.380301 8875 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:58:02.380856 8875 webserver.cc:460] Webserver started at http://127.31.43.254:32815/ using document root <none> and password file <none>
I20260812 03:58:02.381136 8875 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:02.381210 8875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:02.381407 8875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:02.382586 8875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "6af6ce0ba0394f35818211dbece036c1"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "858af19af8c2feb3206386a3cc472bf6"
server_key_iv: "615247f09b75e535b070b16137da27d9"
server_key_version: "encryptionkey@0"
I20260812 03:58:02.383056 8875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "6af6ce0ba0394f35818211dbece036c1"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "858af19af8c2feb3206386a3cc472bf6"
server_key_iv: "615247f09b75e535b070b16137da27d9"
server_key_version: "encryptionkey@0"
I20260812 03:58:02.384522 8875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:02.385414 8893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:02.385671 8875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:02.385783 8875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "6af6ce0ba0394f35818211dbece036c1"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "858af19af8c2feb3206386a3cc472bf6"
server_key_iv: "615247f09b75e535b070b16137da27d9"
server_key_version: "encryptionkey@0"
I20260812 03:58:02.385864 8875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:02.402020 8875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:02.404554 8875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:02.404743 8875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:02.409749 8875 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:42917
I20260812 03:58:02.409816 8953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:42917 every 8 connection(s)
I20260812 03:58:02.410218 8875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:58:02.411360 8954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:58:02.413908 8954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1: Bootstrap starting.
I20260812 03:58:02.414404 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 8875
I20260812 03:58:02.414516 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:58:02.414719 8954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1: Neither blocks nor log segments found. Creating new log.
I20260812 03:58:02.414775 31919 external_mini_cluster.cc:1428] Setting key afa0dbb0d2e8d4990a49ac89e66d01dc
I20260812 03:58:02.415117 8954 log.cc:824] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:58:02.415967 8954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1: No bootstrap required, opened a new log
I20260812 03:58:02.418571 8954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af6ce0ba0394f35818211dbece036c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42917 } }
I20260812 03:58:02.418797 8954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:58:02.418850 8954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6af6ce0ba0394f35818211dbece036c1, State: Initialized, Role: FOLLOWER
I20260812 03:58:02.418994 8954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af6ce0ba0394f35818211dbece036c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42917 } }
I20260812 03:58:02.419080 8954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:58:02.419127 8954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:58:02.419179 8954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:58:02.419945 8954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af6ce0ba0394f35818211dbece036c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42917 } }
I20260812 03:58:02.420070 8954 leader_election.cc:302] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6af6ce0ba0394f35818211dbece036c1; no voters:
I20260812 03:58:02.420272 8954 leader_election.cc:288] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:58:02.420358 8959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:58:02.420570 8954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:58:02.420603 8959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 1 LEADER]: Becoming Leader. State: Replica: 6af6ce0ba0394f35818211dbece036c1, State: Running, Role: LEADER
I20260812 03:58:02.421166 8959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af6ce0ba0394f35818211dbece036c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42917 } }
I20260812 03:58:02.421856 8960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6af6ce0ba0394f35818211dbece036c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af6ce0ba0394f35818211dbece036c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42917 } } }
I20260812 03:58:02.421967 8960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:58:02.422199 8976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:58:02.422194 8961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6af6ce0ba0394f35818211dbece036c1. Latest consensus state: current_term: 1 leader_uuid: "6af6ce0ba0394f35818211dbece036c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af6ce0ba0394f35818211dbece036c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42917 } } }
I20260812 03:58:02.422930 8961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [sys.catalog]: This master's current role is: LEADER
Aug 12 03:58:02 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:58:02.423517 8976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:58:02.425566 8976 catalog_manager.cc:1347] Generated new cluster ID: 0db69d2dbef54d9da63e92fab3275c1c
I20260812 03:58:02.425637 8976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:58:02.427917 8958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:02.415322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45194 (local address 127.31.43.254:42917)
0812 03:58:02.415501 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:02.415512 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:02.415556 (+ 44us) server_negotiation.cc:407] Connection header received
0812 03:58:02.416207 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:02.416217 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:02.416285 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:02.416448 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:02.417181 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.418716 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.419734 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.420432 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.420955 (+ 523us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:02.423827 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:02.423832 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:02.423832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:58:02.423846 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:02.425949 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:02.426384 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:02.426387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:02.426388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:02.426508 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:02.426749 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:02.426750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:02.426750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:02.427097 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:02.427295 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:02.427374 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 03:58:02.427661 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260812 03:58:02.440778 8976 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:58:02.441365 8976 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:58:02.445887 8976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1: Generated new TSK 0
I20260812 03:58:02.446134 8976 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 03:58:02.487874 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42917
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:58:02.579131 8986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:02.579393 8986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:02.579437 8986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:02.580700 8986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:02.580783 8986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:02.580832 8986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:58:02.582553 8986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:02.582897 8986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:02.583175 8986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:02.588749 8995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:02.588761 8993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:02.588761 8992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:02.589221 8986 server_base.cc:1034] running on GCE node
I20260812 03:58:02.589460 8986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:02.589715 8986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:02.590866 8986 hybrid_clock.cc:648] HybridClock initialized: now 1786507082590861 us; error 52 us; skew 500 ppm
Aug 12 03:58:02 dist-test-slave-8tdl krb5kdc[8861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507082, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:58:02.594544 8986 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 03:58:02.595309 8986 webserver.cc:460] Webserver started at http://127.31.43.193:44389/ using document root <none> and password file <none>
I20260812 03:58:02.595569 8986 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:02.595621 8986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:02.595760 8986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:02.596969 8986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "cc0b1826aef840d6b16a4e51ef0796e5"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "6ce694025e8c623feac4dfc0882934d5"
server_key_iv: "de855107ceb9948a2cc15465c3e1e0a4"
server_key_version: "encryptionkey@0"
I20260812 03:58:02.597386 8986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "cc0b1826aef840d6b16a4e51ef0796e5"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "6ce694025e8c623feac4dfc0882934d5"
server_key_iv: "de855107ceb9948a2cc15465c3e1e0a4"
server_key_version: "encryptionkey@0"
I20260812 03:58:02.599023 8986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:58:02.599884 9003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:02.600070 8986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:02.600188 8986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "cc0b1826aef840d6b16a4e51ef0796e5"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "6ce694025e8c623feac4dfc0882934d5"
server_key_iv: "de855107ceb9948a2cc15465c3e1e0a4"
server_key_version: "encryptionkey@0"
I20260812 03:58:02.600271 8986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:02.611232 8986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:02.613485 8986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:02.613679 8986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:02.613950 8986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:02.614310 8986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:02.614369 8986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:02.614416 8986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:02.614442 8986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:02.621359 8986 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:41793
I20260812 03:58:02.621433 9134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:41793 every 8 connection(s)
I20260812 03:58:02.621883 8986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:58:02.625047 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 8986
I20260812 03:58:02.625180 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:58:02.625383 31919 external_mini_cluster.cc:1428] Setting key 46ccbe2874a64815c0eef5eaa2031eff
Aug 12 03:58:02 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507082, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:58:02.633426 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:02.622746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:50265)
0812 03:58:02.622933 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:02.622976 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:02.623467 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:02.623703 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:02.623707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:02.624313 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:02.624531 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:02.624536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.625611 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.625612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:02.626650 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:02.626652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.627202 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.627203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:02.627315 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:02.627946 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:02.627955 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:02.629538 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:58:02.631720 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:58:02.631728 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:58:02.631730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:58:02.632111 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:58:02.632350 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:58:02.632358 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:58:02.632359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:58:02.632472 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:58:02.632818 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:58:02.632824 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:58:02.632956 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 03:58:02.633098 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 03:58:02.633244 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260812 03:58:02.633426 8958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:02.622753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:50265 (local address 127.31.43.254:42917)
0812 03:58:02.622815 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:02.622818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:02.623072 (+ 254us) server_negotiation.cc:407] Connection header received
0812 03:58:02.623586 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:02.623587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:02.623598 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:02.623638 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:02.624656 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.625484 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.626777 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.627110 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.627188 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:02.629661 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:02.629663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:02.629663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:58:02.629678 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:02.631613 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:02.632200 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:02.632201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:02.632202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:02.632275 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:02.632545 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:02.632546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:02.632547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:02.632716 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:02.632812 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:02.633151 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 03:58:02.633271 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:58:02.634212 9135 heartbeater.cc:347] Connected to a master server at 127.31.43.254:42917
I20260812 03:58:02.634322 9135 heartbeater.cc:464] Registering TS with master...
I20260812 03:58:02.634547 9135 heartbeater.cc:511] Master 127.31.43.254:42917 requested a full tablet report, sending...
I20260812 03:58:02.635215 8913 ts_manager.cc:194] Registered new tserver with Master: cc0b1826aef840d6b16a4e51ef0796e5 (127.31.43.193:41793)
I20260812 03:58:02.635857 8913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:50265
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 03:58:02.659328 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42917
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:58:02.750329 9142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:02.750638 9142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:02.750685 9142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:02.752007 9142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:02.752110 9142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:02.752163 9142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:58:02.753871 9142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:02.754222 9142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:02.754500 9142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:02.760383 9149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:02.760378 9151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:02.760382 9148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:02.760793 9142 server_base.cc:1034] running on GCE node
I20260812 03:58:02.761075 9142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:02.761368 9142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:02.762573 9142 hybrid_clock.cc:648] HybridClock initialized: now 1786507082762561 us; error 36 us; skew 500 ppm
Aug 12 03:58:02 dist-test-slave-8tdl krb5kdc[8861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507082, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:58:02.766135 9142 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 03:58:02.766853 9142 webserver.cc:460] Webserver started at http://127.31.43.194:42739/ using document root <none> and password file <none>
I20260812 03:58:02.767149 9142 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:02.767232 9142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:02.767396 9142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:02.768710 9142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "0c96ca1ae3b14e7a8468d7689f6a6fdb"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "0eb5c7df0a2e738ab9c0cacc4a3fb18f"
server_key_iv: "26484ad00393da05afb8667dcaa830c0"
server_key_version: "encryptionkey@0"
I20260812 03:58:02.769150 9142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "0c96ca1ae3b14e7a8468d7689f6a6fdb"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "0eb5c7df0a2e738ab9c0cacc4a3fb18f"
server_key_iv: "26484ad00393da05afb8667dcaa830c0"
server_key_version: "encryptionkey@0"
I20260812 03:58:02.770798 9142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 03:58:02.771828 9158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:02.772080 9142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:02.772202 9142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "0c96ca1ae3b14e7a8468d7689f6a6fdb"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "0eb5c7df0a2e738ab9c0cacc4a3fb18f"
server_key_iv: "26484ad00393da05afb8667dcaa830c0"
server_key_version: "encryptionkey@0"
I20260812 03:58:02.772286 9142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:02.780965 9142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:02.783874 9142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:02.784085 9142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:02.784369 9142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:02.784737 9142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:02.784794 9142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:02.784833 9142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:02.784883 9142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:02.792078 9142 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:42839
I20260812 03:58:02.792166 9284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:42839 every 8 connection(s)
I20260812 03:58:02.792553 9142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:58:02.796751 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 9142
I20260812 03:58:02.796995 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:58:02.797330 31919 external_mini_cluster.cc:1428] Setting key 249fedf5200459a093eae0e660159ba5
Aug 12 03:58:02 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507082, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:58:02.805158 8958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:02.793756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:46277 (local address 127.31.43.254:42917)
0812 03:58:02.793841 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:02.793844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:02.794125 (+ 281us) server_negotiation.cc:407] Connection header received
0812 03:58:02.794717 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:02.794719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:02.794734 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:02.794812 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:02.795925 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.796562 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.798088 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.798373 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.798432 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:02.801763 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:02.801765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:02.801765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:58:02.801780 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:02.803447 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:02.804028 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:02.804034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:02.804034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:02.804097 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:02.804408 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:02.804409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:02.804410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:02.804552 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:02.804637 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:02.804958 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 03:58:02.805055 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:58:02.805439 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:02.793715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:46277)
0812 03:58:02.793972 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:02.794019 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:02.794553 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:02.794890 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:02.794895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:02.795566 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:02.795784 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:02.795791 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.796739 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.796741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:02.797939 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:02.797942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.798486 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.798488 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:02.798667 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:02.799376 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:02.799390 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:02.801631 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:58:02.803591 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:58:02.803603 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:58:02.803606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:58:02.803894 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:58:02.804177 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:58:02.804178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:58:02.804178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:58:02.804345 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:58:02.804628 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:58:02.804637 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:58:02.804806 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 03:58:02.804921 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:58:02.805124 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":127,"threads_started":1}
I20260812 03:58:02.805984 9285 heartbeater.cc:347] Connected to a master server at 127.31.43.254:42917
I20260812 03:58:02.806099 9285 heartbeater.cc:464] Registering TS with master...
I20260812 03:58:02.806313 9285 heartbeater.cc:511] Master 127.31.43.254:42917 requested a full tablet report, sending...
I20260812 03:58:02.806847 8913 ts_manager.cc:194] Registered new tserver with Master: 0c96ca1ae3b14e7a8468d7689f6a6fdb (127.31.43.194:42839)
I20260812 03:58:02.807299 8913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:46277
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 03:58:02.829970 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42917
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:58:02.919664 9290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:02.919902 9290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:02.919929 9290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:02.921278 9290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:02.921326 9290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:02.921353 9290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:58:02.923234 9290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:02.923509 9290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:02.923751 9290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:02.929328 9295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:02.929585 9290 server_base.cc:1034] running on GCE node
W20260812 03:58:02.929318 9299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:02.929323 9296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:02.929921 9290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:02.930235 9290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:02.931411 9290 hybrid_clock.cc:648] HybridClock initialized: now 1786507082931400 us; error 49 us; skew 500 ppm
Aug 12 03:58:02 dist-test-slave-8tdl krb5kdc[8861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507082, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:58:02.934338 9290 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 03:58:02.934958 9290 webserver.cc:460] Webserver started at http://127.31.43.195:43941/ using document root <none> and password file <none>
I20260812 03:58:02.935230 9290 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:02.935307 9290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:02.935495 9290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:02.936805 9290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "74d2581331164e8b8ddb4696e6eaa34b"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "2a4baa8616d8a7d461223a82ec452271"
server_key_iv: "5b04a202ec4fa88c011a8b256d9731e2"
server_key_version: "encryptionkey@0"
I20260812 03:58:02.937242 9290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "74d2581331164e8b8ddb4696e6eaa34b"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "2a4baa8616d8a7d461223a82ec452271"
server_key_iv: "5b04a202ec4fa88c011a8b256d9731e2"
server_key_version: "encryptionkey@0"
I20260812 03:58:02.938892 9290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 03:58:02.939924 9306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:02.940217 9290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:02.940356 9290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "74d2581331164e8b8ddb4696e6eaa34b"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "2a4baa8616d8a7d461223a82ec452271"
server_key_iv: "5b04a202ec4fa88c011a8b256d9731e2"
server_key_version: "encryptionkey@0"
I20260812 03:58:02.940454 9290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:02.948611 9290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:02.950963 9290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:02.951184 9290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:02.951447 9290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:02.951776 9290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:02.951831 9290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:02.951875 9290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:02.951900 9290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:02.958222 9290 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:36099
I20260812 03:58:02.958261 9430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:36099 every 8 connection(s)
I20260812 03:58:02.958607 9290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:58:02 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507082, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:58:02.966845 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 9290
I20260812 03:58:02.966962 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:58:02.967204 31919 external_mini_cluster.cc:1428] Setting key 006180ac3cf28dfe4b0810a8c66f085b
I20260812 03:58:02.969964 8958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:02.959475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:54827 (local address 127.31.43.254:42917)
0812 03:58:02.959574 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:02.959577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:02.959723 (+ 146us) server_negotiation.cc:407] Connection header received
0812 03:58:02.960252 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:02.960253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:02.960263 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:02.960320 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:02.961408 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.962043 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.963312 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.963550 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.963619 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:02.966141 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:02.966142 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:02.966142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:58:02.966155 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:02.968408 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:02.968892 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:02.968894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:02.968894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:02.968965 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:02.969342 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:02.969343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:02.969343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:02.969467 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:02.969526 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:02.969814 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 03:58:02.969888 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:58:02.970284 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:02.959418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:54827)
0812 03:58:02.959597 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:02.959635 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:02.960158 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:02.960389 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:02.960392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:02.961044 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:02.961277 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:02.961283 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.962174 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.962175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:02.963202 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:02.963205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:02.963656 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:02.963657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:02.963775 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:02.964400 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:02.964409 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:02.965981 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:58:02.968513 (+ 2532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:58:02.968522 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:58:02.968524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:58:02.968780 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:58:02.969130 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:58:02.969131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:58:02.969132 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:58:02.969270 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:58:02.969540 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:58:02.969547 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:58:02.969688 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 03:58:02.969785 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:58:02.969949 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":109,"threads_started":1}
I20260812 03:58:02.970932 9431 heartbeater.cc:347] Connected to a master server at 127.31.43.254:42917
I20260812 03:58:02.971045 9431 heartbeater.cc:464] Registering TS with master...
I20260812 03:58:02.971258 9431 heartbeater.cc:511] Master 127.31.43.254:42917 requested a full tablet report, sending...
I20260812 03:58:02.971771 8913 ts_manager.cc:194] Registered new tserver with Master: 74d2581331164e8b8ddb4696e6eaa34b (127.31.43.195:36099)
I20260812 03:58:02.972241 8913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:54827
I20260812 03:58:02.979544 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab.
Entry for principal oryx/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab.
WARNING: no policy specified for oryx/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab.
Entry for principal oryx/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab.
WARNING: no policy specified for oryx/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab.
Entry for principal oryx/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab.
I20260812 03:58:03.074115 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 8986
I20260812 03:58:03.082859 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 9142
I20260812 03:58:03.095652 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 9290
I20260812 03:58:03.104868 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 8875
I20260812 03:58:03.114012 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:42917
--webserver_interface=127.31.43.254
--webserver_port=32815
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:42917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:58:03.198326 9440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:03.198606 9440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:03.198647 9440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:03.199875 9440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:58:03.199957 9440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:03.199983 9440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:03.200016 9440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:58:03.200047 9440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:58:03.201632 9440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:42917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:42917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=32815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:03.201957 9440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:03.202215 9440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:03.207578 9446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:03.207521 9445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:03.207671 9440 server_base.cc:1034] running on GCE node
W20260812 03:58:03.207516 9448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:03.208127 9440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:03.208442 9440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:03.209645 9440 hybrid_clock.cc:648] HybridClock initialized: now 1786507083209612 us; error 74 us; skew 500 ppm
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507083, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:58:03.212769 9440 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:58:03.213289 9440 webserver.cc:460] Webserver started at http://127.31.43.254:32815/ using document root <none> and password file <none>
I20260812 03:58:03.213584 9440 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:03.213680 9440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:03.215462 9440 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:58:03.216290 9456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:03.216503 9440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:58:03.216637 9440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "6af6ce0ba0394f35818211dbece036c1"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "858af19af8c2feb3206386a3cc472bf6"
server_key_iv: "615247f09b75e535b070b16137da27d9"
server_key_version: "encryptionkey@0"
I20260812 03:58:03.217097 9440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:03.238858 9440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:03.241365 9440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:03.241524 9440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:03.246002 9440 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:42917
I20260812 03:58:03.246052 9513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:42917 every 8 connection(s)
I20260812 03:58:03.246518 9440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:58:03.248499 9514 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:58:03.249305 9514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1: Bootstrap starting.
I20260812 03:58:03.250010 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 9440
I20260812 03:58:03.250373 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:41793
--local_ip_for_outbound_sockets=127.31.43.193
--tserver_master_addrs=127.31.43.254:42917
--webserver_port=44389
--webserver_interface=127.31.43.193
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.43.193
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:58:03.260419 9514 log.cc:824] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:58:03.262537 9514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:58:03.262971 9514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1: Bootstrap complete.
I20260812 03:58:03.265772 9514 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af6ce0ba0394f35818211dbece036c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42917 } }
I20260812 03:58:03.266057 9514 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6af6ce0ba0394f35818211dbece036c1, State: Initialized, Role: FOLLOWER
I20260812 03:58:03.266296 9514 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af6ce0ba0394f35818211dbece036c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42917 } }
I20260812 03:58:03.266413 9514 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:58:03.266474 9514 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:58:03.266530 9514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:58:03.267750 9514 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af6ce0ba0394f35818211dbece036c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42917 } }
I20260812 03:58:03.267896 9514 leader_election.cc:302] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6af6ce0ba0394f35818211dbece036c1; no voters:
I20260812 03:58:03.268152 9514 leader_election.cc:288] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:58:03.268194 9518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:58:03.268332 9518 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [term 2 LEADER]: Becoming Leader. State: Replica: 6af6ce0ba0394f35818211dbece036c1, State: Running, Role: LEADER
I20260812 03:58:03.268478 9514 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:58:03.268515 9518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af6ce0ba0394f35818211dbece036c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42917 } }
I20260812 03:58:03.268994 9520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6af6ce0ba0394f35818211dbece036c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af6ce0ba0394f35818211dbece036c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42917 } } }
I20260812 03:58:03.269115 9520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:58:03.269585 9519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6af6ce0ba0394f35818211dbece036c1. Latest consensus state: current_term: 2 leader_uuid: "6af6ce0ba0394f35818211dbece036c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af6ce0ba0394f35818211dbece036c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42917 } } }
I20260812 03:58:03.269677 9519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:58:03.269863 9526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:58:03.270475 9526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:58:03.270742 9526 catalog_manager.cc:1259] Loaded cluster ID: 0db69d2dbef54d9da63e92fab3275c1c
I20260812 03:58:03.270982 9526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:58:03.272488 9526 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:58:03.272773 9526 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6af6ce0ba0394f35818211dbece036c1: Loaded TSK: 0
I20260812 03:58:03.273058 9526 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:58:03.348847 9516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:03.349134 9516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:03.349185 9516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:03.350582 9516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:03.350664 9516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:03.350747 9516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:58:03.352496 9516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.193.keytab
--principal=oryx/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:41793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=44389
--tserver_master_addrs=127.31.43.254:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:03.352823 9516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:03.353164 9516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:03.359916 9545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:03.359949 9543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:03.359910 9542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:03.360170 9516 server_base.cc:1034] running on GCE node
I20260812 03:58:03.360450 9516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:03.360800 9516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:03.362002 9516 hybrid_clock.cc:648] HybridClock initialized: now 1786507083361980 us; error 50 us; skew 500 ppm
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507083, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:58:03.365233 9516 init.cc:377] Logged in from keytab as oryx/127.31.43.193@KRBTEST.COM (short username oryx)
I20260812 03:58:03.365779 9516 webserver.cc:460] Webserver started at http://127.31.43.193:44389/ using document root <none> and password file <none>
I20260812 03:58:03.366036 9516 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:03.366129 9516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:03.367779 9516 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:03.368515 9554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:03.368697 9516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:03.368816 9516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "cc0b1826aef840d6b16a4e51ef0796e5"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "6ce694025e8c623feac4dfc0882934d5"
server_key_iv: "de855107ceb9948a2cc15465c3e1e0a4"
server_key_version: "encryptionkey@0"
I20260812 03:58:03.369313 9516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:03.402194 9516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:03.405273 9516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:03.405462 9516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:03.405778 9516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:03.406190 9516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:03.406239 9516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:03.406275 9516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:03.406294 9516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:03.415122 9516 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:41793
I20260812 03:58:03.415162 9678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:41793 every 8 connection(s)
I20260812 03:58:03.415647 9516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:58:03.418387 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 9516
I20260812 03:58:03.418799 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:42839
--local_ip_for_outbound_sockets=127.31.43.194
--tserver_master_addrs=127.31.43.254:42917
--webserver_port=42739
--webserver_interface=127.31.43.194
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.43.194
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:03.427629 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:03.416654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:52367)
0812 03:58:03.416862 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:03.416918 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:03.417467 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:03.417966 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:03.417970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:03.419037 (+ 1067us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:03.419336 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.419345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.421286 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.421287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.422559 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.422563 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.423471 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.423472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.423596 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.424200 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:03.424209 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:03.427311 (+ 3102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
W20260812 03:58:03.427659 9683 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:03.416711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:52367 (local address 127.31.43.254:42917)
0812 03:58:03.416948 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:03.416956 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:03.417024 (+ 68us) server_negotiation.cc:407] Connection header received
0812 03:58:03.417734 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:03.417739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:03.417785 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:03.417924 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:03.419508 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.421115 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.422730 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.423366 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.423501 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.427504 (+ 4003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:52367: BlockingRecv error: recv got EOF from 127.31.43.193:52367 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":112,"threads_started":1}
W20260812 03:58:03.428090 9679 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:42917 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.43.254:42917: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:03.433770 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:03.428416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:53149)
0812 03:58:03.428601 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:03.428614 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:03.428660 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:03.428943 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:03.428945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:03.429131 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:03.429184 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.429186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.429875 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.429876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.430385 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.430386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.430817 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.430818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.430907 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.431393 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:03.431398 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:03.433682 (+ 2284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159}
W20260812 03:58:03.433949 9685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:03.428553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:53149 (local address 127.31.43.254:42917)
0812 03:58:03.428789 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:03.428791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:03.428800 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:58:03.428826 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:03.428827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:03.428840 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:03.428937 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:03.429283 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.429789 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.430472 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.430722 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.430817 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.433842 (+ 3025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:53149: BlockingRecv error: recv got EOF from 127.31.43.193:53149 (error 108)
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":131,"threads_started":1}
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:03.439217 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:03.434193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:57979)
0812 03:58:03.434249 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:03.434258 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:03.434313 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:03.434458 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:03.434459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:03.434601 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:03.434632 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.434634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.435754 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.435755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.436302 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.436305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.436844 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.436844 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.436884 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.437345 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:03.437350 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:03.439128 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 03:58:03.439388 9685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:03.434252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:57979 (local address 127.31.43.254:42917)
0812 03:58:03.434311 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:03.434313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:03.434322 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:58:03.434371 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:03.434371 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:03.434389 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:03.434454 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:03.434763 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.435658 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.436410 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.436766 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.436842 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.439230 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:57979: BlockingRecv error: recv got EOF from 127.31.43.193:57979 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 03:58:03.439723 9679 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:58:03.512346 9684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:03.512580 9684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:03.512606 9684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:03.513870 9684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:03.513917 9684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:03.513952 9684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:58:03.515548 9684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.194.keytab
--principal=oryx/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:42839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=42739
--tserver_master_addrs=127.31.43.254:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:03.515833 9684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:03.516116 9684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:03.521690 9691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:03.521706 9693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:03.521760 9690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:03.521893 9684 server_base.cc:1034] running on GCE node
I20260812 03:58:03.522213 9684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:03.522462 9684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:03.523622 9684 hybrid_clock.cc:648] HybridClock initialized: now 1786507083523580 us; error 59 us; skew 500 ppm
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507083, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:58:03.529865 9684 init.cc:377] Logged in from keytab as oryx/127.31.43.194@KRBTEST.COM (short username oryx)
I20260812 03:58:03.530403 9684 webserver.cc:460] Webserver started at http://127.31.43.194:42739/ using document root <none> and password file <none>
I20260812 03:58:03.530725 9684 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:03.530812 9684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:03.532426 9684 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:03.533123 9704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:03.533332 9684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:03.533452 9684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "0c96ca1ae3b14e7a8468d7689f6a6fdb"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "0eb5c7df0a2e738ab9c0cacc4a3fb18f"
server_key_iv: "26484ad00393da05afb8667dcaa830c0"
server_key_version: "encryptionkey@0"
I20260812 03:58:03.533891 9684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:03.557828 9684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:03.560694 9684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:03.560904 9684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:03.561192 9684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:03.561578 9684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:03.561648 9684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:03.561690 9684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:03.561744 9684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:03.568984 9684 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:42839
I20260812 03:58:03.569103 9838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:42839 every 8 connection(s)
I20260812 03:58:03.569501 9684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:03.577124 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 9684
I20260812 03:58:03.577440 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:36099
--local_ip_for_outbound_sockets=127.31.43.195
--tserver_master_addrs=127.31.43.254:42917
--webserver_port=43941
--webserver_interface=127.31.43.195
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.43.195
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:58:03.578307 9685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:03.570434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:47781 (local address 127.31.43.254:42917)
0812 03:58:03.570521 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:03.570523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:03.570763 (+ 240us) server_negotiation.cc:407] Connection header received
0812 03:58:03.571327 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:03.571329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:03.571342 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:03.571410 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:03.572603 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.573234 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.574363 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.574576 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.574644 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.578255 (+ 3611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:47781: BlockingRecv error: recv got EOF from 127.31.43.194:47781 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:58:03.578384 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:03.570408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:47781)
0812 03:58:03.570594 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:03.570643 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:03.571223 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:03.571470 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:03.571474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:03.572163 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:03.572487 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.572494 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.573356 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.573357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.574260 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.574264 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.574695 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.574696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.574819 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.575379 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:03.575388 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:03.578128 (+ 2740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
W20260812 03:58:03.578649 9839 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:42917 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.43.254:42917: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:03.583894 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:03.579013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:58749)
0812 03:58:03.579069 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:03.579078 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:03.579134 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:03.579290 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:03.579291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:03.579446 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:03.579498 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.579499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.580238 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.580240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.580755 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.580757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.581129 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.581130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.581197 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.581674 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:03.581679 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:03.583829 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 03:58:03.584062 9685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:03.579070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:58749 (local address 127.31.43.254:42917)
0812 03:58:03.579134 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:03.579136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:03.579145 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:58:03.579195 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:03.579196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:03.579209 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:03.579284 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:03.579589 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.580138 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.580844 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.581051 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.581136 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.583974 (+ 2838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:58749: BlockingRecv error: recv got EOF from 127.31.43.194:58749 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:03.590209 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:03.584357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:34349)
0812 03:58:03.584529 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:03.584549 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:03.584615 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:03.584758 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:03.584759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:03.584933 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:03.584983 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.584985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.585705 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.585707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.586625 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.586628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.587065 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.587066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.587137 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.587642 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:03.587649 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:03.590089 (+ 2440us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151}
W20260812 03:58:03.590274 9685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:03.584429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:34349 (local address 127.31.43.254:42917)
0812 03:58:03.584486 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:03.584488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:03.584624 (+ 136us) server_negotiation.cc:407] Connection header received
0812 03:58:03.584685 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:03.584685 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:03.584696 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:03.584748 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:03.585061 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.585613 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.586756 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.586973 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.587035 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.590202 (+ 3167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:34349: BlockingRecv error: recv got EOF from 127.31.43.194:34349 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 03:58:03.590394 9839 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:58:03.670533 9843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:03.670832 9843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:03.670876 9843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:03.672307 9843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:03.672384 9843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:03.672435 9843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:58:03.674263 9843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/krb5kdc/oryx_127.31.43.195.keytab
--principal=oryx/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:36099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=43941
--tserver_master_addrs=127.31.43.254:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:03.674573 9843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:03.674919 9843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:03.680430 9853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:03.680433 9850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:03.680469 9851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:03.680757 9843 server_base.cc:1034] running on GCE node
I20260812 03:58:03.681066 9843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:03.681416 9843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:03.682612 9843 hybrid_clock.cc:648] HybridClock initialized: now 1786507083682578 us; error 51 us; skew 500 ppm
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507083, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:58:03.685541 9843 init.cc:377] Logged in from keytab as oryx/127.31.43.195@KRBTEST.COM (short username oryx)
I20260812 03:58:03.685986 9843 webserver.cc:460] Webserver started at http://127.31.43.195:43941/ using document root <none> and password file <none>
I20260812 03:58:03.686204 9843 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:03.686260 9843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:03.687734 9843 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:03.688308 9861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:03.688512 9843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:03.688622 9843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "74d2581331164e8b8ddb4696e6eaa34b"
format_stamp: "Formatted at 2026-08-12 03:58:02 on dist-test-slave-8tdl"
server_key: "2a4baa8616d8a7d461223a82ec452271"
server_key_iv: "5b04a202ec4fa88c011a8b256d9731e2"
server_key_version: "encryptionkey@0"
I20260812 03:58:03.688993 9843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:03.704696 9843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:03.707448 9843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:03.707590 9843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:03.707832 9843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:03.708227 9843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:03.708269 9843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:03.708293 9843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:03.708307 9843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:03.714648 9843 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:36099
I20260812 03:58:03.714756 9987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:36099 every 8 connection(s)
I20260812 03:58:03.715175 9843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:03.724293 9991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:03.715987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:44997)
0812 03:58:03.716193 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:03.716249 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:03.716783 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:03.717021 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:03.717025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:03.717662 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:03.717871 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.717877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.718586 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.718587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.719504 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.719507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.719967 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.719972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.720094 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.720775 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:03.720788 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:03.724018 (+ 3230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
W20260812 03:58:03.724291 9685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:03.716001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:44997 (local address 127.31.43.254:42917)
0812 03:58:03.716077 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:03.716079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:03.716360 (+ 281us) server_negotiation.cc:407] Connection header received
0812 03:58:03.716877 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:03.716878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:03.716889 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:03.716947 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:03.717981 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.718491 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.719598 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.719829 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.719905 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.724180 (+ 4275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:44997: BlockingRecv error: recv got EOF from 127.31.43.195:44997 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:58:03.724632 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 9843
W20260812 03:58:03.725196 9988 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:42917 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.43.254:42917: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:03.731920 9991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:03.726561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:54591)
0812 03:58:03.726639 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:03.726650 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:03.726751 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:03.726955 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:03.726956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:03.727135 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:03.727200 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.727202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.728107 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.728108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.728620 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.728622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.729028 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.729029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.729096 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.729699 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:03.729703 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:03.731860 (+ 2157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 03:58:03.732059 9683 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:03.726559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:54591 (local address 127.31.43.254:42917)
0812 03:58:03.726605 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:03.726607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:03.726722 (+ 115us) server_negotiation.cc:407] Connection header received
0812 03:58:03.726829 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:03.726830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:03.726845 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:03.726887 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:03.727272 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.728022 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.728727 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.728951 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.729030 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.731995 (+ 2965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:54591: BlockingRecv error: recv got EOF from 127.31.43.195:54591 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:58:03.734652 9685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:03.725450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45198 (local address 127.31.43.254:42917)
0812 03:58:03.725539 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:03.725541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:03.725617 (+ 76us) server_negotiation.cc:407] Connection header received
0812 03:58:03.725749 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:03.725750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:03.725763 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:03.725832 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:03.726778 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.727559 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.728664 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.728883 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.728975 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.730600 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:03.730605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:03.730606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:58:03.730620 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:03.732939 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:03.733389 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:03.733392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:03.733392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:03.733495 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:03.733730 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:03.733731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:03.733731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:03.734138 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:03.734285 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:03.734400 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 03:58:03.734517 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:03 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:03.737594 9991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:03.732381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:58767)
0812 03:58:03.732429 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:03.732438 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:03.732475 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:03.732610 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:03.732610 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:03.732757 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:03.732803 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.732805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.733733 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.733734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.734489 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:03.734492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.735096 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.735097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:03.735155 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.735612 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:03.735617 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:03.737537 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 03:58:03.737757 9683 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:03.732452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:58767 (local address 127.31.43.254:42917)
0812 03:58:03.732493 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:03.732495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:03.732521 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:58:03.732551 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:03.732551 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:03.732560 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:03.732600 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:03.732879 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.733628 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.734651 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:03.735015 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:03.735092 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:03.737687 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:58767: BlockingRecv error: recv got EOF from 127.31.43.195:58767 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 03:58:03.737758 9988 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 03:58:04 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:04 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:04.451185 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:04.440273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:44107)
0812 03:58:04.440474 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:04.440493 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:04.440574 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:04.440766 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:04.440766 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:04.440946 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:04.441003 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:04.441005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:04.441780 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:04.441781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:04.442467 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:04.442471 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:04.442944 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:04.442945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:04.443017 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:04.448935 (+ 5918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:04.448944 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:04.451095 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":98,"threads_started":1}
W20260812 03:58:04.451390 9994 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:04.440418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:44107 (local address 127.31.43.254:42917)
0812 03:58:04.440598 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:04.440600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:04.440612 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:58:04.440645 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:04.440645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:04.440657 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:04.440745 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:04.441116 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:04.441657 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:04.442598 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:04.442843 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:04.442913 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:04.451302 (+ 8389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:44107: BlockingRecv error: recv got EOF from 127.31.43.193:44107 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":80,"threads_started":1}
Aug 12 03:58:04 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:04 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:04.596618 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:04.590985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:35813)
0812 03:58:04.591168 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:04.591179 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:04.591229 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:04.591361 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:04.591362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:04.591541 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:04.591595 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:04.591597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:04.592434 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:04.592435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:04.593142 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:04.593145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:04.593657 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:04.593658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:04.593722 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:04.594304 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:04.594309 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:04.596545 (+ 2236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":115,"threads_started":1}
W20260812 03:58:04.596810 9994 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:04.591125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:35813 (local address 127.31.43.254:42917)
0812 03:58:04.591205 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:04.591207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:04.591250 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:58:04.591281 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:04.591282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:04.591293 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:04.591341 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:04.591693 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:04.592344 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:04.593240 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:04.593572 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:04.593647 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:04.596731 (+ 3084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:35813: BlockingRecv error: recv got EOF from 127.31.43.194:35813 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 03:58:04 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:04 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:04.744210 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:04.738277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:33439)
0812 03:58:04.738495 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:04.738544 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:04.738656 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:04.738914 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:04.738915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:04.739137 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:04.739280 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:04.739285 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:04.740152 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:04.740153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:04.740763 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:04.740766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:04.741244 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:04.741245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:04.741296 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:04.741809 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:04.741815 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:04.744132 (+ 2317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":106,"threads_started":1}
W20260812 03:58:04.744288 9994 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:04.738401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:33439 (local address 127.31.43.254:42917)
0812 03:58:04.738458 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:04.738461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:04.738623 (+ 162us) server_negotiation.cc:407] Connection header received
0812 03:58:04.738810 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:04.738811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:04.738825 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:04.738885 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:04.739392 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:04.740038 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:04.740873 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:04.741067 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:04.741228 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:04.744238 (+ 3010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:33439: BlockingRecv error: recv got EOF from 127.31.43.195:33439 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 12 03:58:05 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:05 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:05.456974 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:05.451858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:52345)
0812 03:58:05.452026 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:05.452045 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:05.452099 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:05.452331 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:05.452332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:05.452515 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:05.452570 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:05.452571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:05.453313 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:05.453314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:05.453869 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:05.453870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:05.454277 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:05.454277 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:05.454324 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:05.454893 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:05.454899 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:05.456882 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
W20260812 03:58:05.457098 10000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:05.451969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:52345 (local address 127.31.43.254:42917)
0812 03:58:05.452172 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:05.452179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:05.452189 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:58:05.452218 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:05.452219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:05.452229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:05.452309 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:05.452669 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:05.453221 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:05.453965 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:05.454195 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:05.454255 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:05.457013 (+ 2758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:52345: BlockingRecv error: recv got EOF from 127.31.43.193:52345 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
Aug 12 03:58:05 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:05 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:05.602921 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:05.597280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:49871)
0812 03:58:05.597420 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:05.597434 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:05.597524 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:05.597692 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:05.597693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:05.597985 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:05.598045 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:05.598047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:05.598958 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:05.598960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:05.599560 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:05.599561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:05.600028 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:05.600030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:05.600097 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:05.600648 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:05.600654 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:05.602831 (+ 2177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
W20260812 03:58:05.603122 10000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:05.597417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:49871 (local address 127.31.43.254:42917)
0812 03:58:05.597498 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:05.597501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:05.597514 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:58:05.597607 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:05.597608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:05.597623 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:05.597674 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:05.598153 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:05.598859 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:05.599662 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:05.599922 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:05.600030 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:05.603051 (+ 3021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:49871: BlockingRecv error: recv got EOF from 127.31.43.194:49871 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 03:58:05 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:05 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:05.750830 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:05.744818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:41909)
0812 03:58:05.744993 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:05.745009 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:05.745093 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:05.745300 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:05.745301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:05.745501 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:05.745570 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:05.745572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:05.746376 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:05.746377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:05.747048 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:05.747051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:05.747475 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:05.747477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:05.747552 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:05.748168 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:05.748175 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:05.750708 (+ 2533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":120,"threads_started":1}
W20260812 03:58:05.750914 10000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:05.744959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:41909 (local address 127.31.43.254:42917)
0812 03:58:05.745020 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:05.745023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:05.745070 (+ 47us) server_negotiation.cc:407] Connection header received
0812 03:58:05.745189 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:05.745190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:05.745207 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:05.745291 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:05.745669 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:05.746270 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:05.747156 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:05.747376 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:05.747462 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:05.750835 (+ 3373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:41909: BlockingRecv error: recv got EOF from 127.31.43.195:41909 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 12 03:58:06 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:06 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:06.462458 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:06.457451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:55825)
0812 03:58:06.457617 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:06.457628 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:06.457691 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:06.457851 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:06.457852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:06.458026 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:06.458086 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:06.458088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:06.458864 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:06.458865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:06.459442 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:06.459444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:06.459860 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:06.459861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:06.459916 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:06.460413 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:06.460418 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:06.462381 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
W20260812 03:58:06.462623 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:06.457569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:55825 (local address 127.31.43.254:42917)
0812 03:58:06.457711 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:06.457713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:06.457723 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:58:06.457754 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:06.457755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:06.457764 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:06.457831 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:06.458179 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:06.458788 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:06.459553 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:06.459759 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:06.459824 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:06.462532 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:55825: BlockingRecv error: recv got EOF from 127.31.43.193:55825 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
Aug 12 03:58:06 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:06 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:06.610015 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:06.603573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:41599)
0812 03:58:06.603796 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:06.603810 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:06.603912 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:06.604111 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:06.604112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:06.604328 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:06.604381 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:06.604384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:06.605249 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:06.605250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:06.605891 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:06.605893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:06.606409 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:06.606409 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:06.606491 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:06.607180 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:06.607186 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:06.609895 (+ 2709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":110,"threads_started":1}
W20260812 03:58:06.610106 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:06.603686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:41599 (local address 127.31.43.254:42917)
0812 03:58:06.603768 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:06.603771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:06.603887 (+ 116us) server_negotiation.cc:407] Connection header received
0812 03:58:06.603991 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:06.603992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:06.604006 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:06.604087 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:06.604544 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:06.605123 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:06.606075 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:06.606311 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:06.606397 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:06.610037 (+ 3640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:41599: BlockingRecv error: recv got EOF from 127.31.43.194:41599 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 03:58:06 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:06 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:06.757892 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:06.751625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:55153)
0812 03:58:06.752042 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:06.752053 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:06.752224 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:06.752458 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:06.752459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:06.752677 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:06.752739 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:06.752741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:06.753535 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:06.753537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:06.754115 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:06.754117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:06.754530 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:06.754531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:06.754599 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:06.755287 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:06.755292 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:06.757753 (+ 2461us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":155,"threads_started":1}
W20260812 03:58:06.757990 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:06.751796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:55153 (local address 127.31.43.254:42917)
0812 03:58:06.751931 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:06.751934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:06.752191 (+ 257us) server_negotiation.cc:407] Connection header received
0812 03:58:06.752322 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:06.752323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:06.752349 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:06.752444 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:06.752842 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:06.753410 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:06.754225 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:06.754441 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:06.754528 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:06.757891 (+ 3363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:55153: BlockingRecv error: recv got EOF from 127.31.43.195:55153 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 03:58:07 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:07 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:07.468637 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:07.463150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:36087)
0812 03:58:07.463307 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:07.463325 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:07.463383 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:07.463551 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:07.463552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:07.463736 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:07.463793 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:07.463795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:07.464537 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:07.464539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:07.465106 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:07.465108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:07.465575 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:07.465576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:07.465626 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:07.466217 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:07.466222 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:07.468570 (+ 2348us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
W20260812 03:58:07.468787 10008 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:07.463225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:36087 (local address 127.31.43.254:42917)
0812 03:58:07.463379 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:07.463381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:07.463393 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:58:07.463426 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:07.463427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:07.463437 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:07.463507 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:07.463885 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:07.464427 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:07.465237 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:07.465490 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:07.465560 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:07.468679 (+ 3119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:36087: BlockingRecv error: recv got EOF from 127.31.43.193:36087 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
Aug 12 03:58:07 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:07 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:07.616411 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:07.610603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:58209)
0812 03:58:07.610818 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:07.610830 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:07.610897 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:07.611220 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:07.611221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:07.611440 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:07.611507 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:07.611509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:07.612522 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:07.612524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:07.613326 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:07.613328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:07.613839 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:07.613840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:07.613905 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:07.614503 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:07.614509 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:07.616347 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":115,"threads_started":1}
W20260812 03:58:07.616585 10008 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:07.610763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:58209 (local address 127.31.43.254:42917)
0812 03:58:07.610822 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:07.610824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:07.610896 (+ 72us) server_negotiation.cc:407] Connection header received
0812 03:58:07.611006 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:07.611007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:07.611021 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:07.611191 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:07.611641 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:07.612414 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:07.613415 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:07.613735 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:07.613812 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:07.616484 (+ 2672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:58209: BlockingRecv error: recv got EOF from 127.31.43.194:58209 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 12 03:58:07 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:07 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:07.764725 10011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:07.759044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:58887)
0812 03:58:07.759217 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:07.759229 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:07.759283 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:07.759476 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:07.759477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:07.759674 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:07.759727 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:07.759730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:07.760642 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:07.760643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:07.761211 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:07.761213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:07.761663 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:07.761664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:07.761720 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:07.762343 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:07.762348 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:07.764658 (+ 2310us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
W20260812 03:58:07.764868 10008 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:07.759213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:58887 (local address 127.31.43.254:42917)
0812 03:58:07.759283 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:07.759285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:07.759320 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:58:07.759344 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:07.759345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:07.759358 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:07.759402 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:07.759851 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:07.760554 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:07.761334 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:07.761555 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:07.761630 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:07.764773 (+ 3143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:58887: BlockingRecv error: recv got EOF from 127.31.43.195:58887 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 03:58:08 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:08 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:08.475082 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:08.469257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:59739)
0812 03:58:08.469433 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:08.469447 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:08.469530 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:08.469749 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:08.469749 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:08.469923 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:08.470044 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:08.470046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:08.470795 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:08.470796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:08.471398 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:08.471401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:08.471869 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:08.471870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:08.471934 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:08.472566 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:08.472570 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:08.475008 (+ 2438us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
W20260812 03:58:08.475292 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:08.469392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:59739 (local address 127.31.43.254:42917)
0812 03:58:08.469614 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:08.469616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:08.469628 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:58:08.469664 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:08.469665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:08.469676 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:08.469736 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:08.470147 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:08.470671 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:08.471492 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:08.471773 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:08.471833 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:08.475191 (+ 3358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:59739: BlockingRecv error: recv got EOF from 127.31.43.193:59739 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":110,"threads_started":1}
Aug 12 03:58:08 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:08 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:08.622972 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:08.617012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:39635)
0812 03:58:08.617570 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:08.617601 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:08.617683 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:08.617848 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:08.617849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:08.618062 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:08.618117 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:08.618120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:08.618907 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:08.618908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:08.619500 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:08.619502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:08.619905 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:08.619906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:08.619961 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:08.620450 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:08.620454 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:08.622886 (+ 2432us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":75,"threads_started":1}
W20260812 03:58:08.623150 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:08.617156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:39635 (local address 127.31.43.254:42917)
0812 03:58:08.617218 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:08.617221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:08.617678 (+ 457us) server_negotiation.cc:407] Connection header received
0812 03:58:08.617760 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:08.617761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:08.617775 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:08.617836 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:08.618222 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:08.618812 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:08.619588 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:08.619841 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:08.619906 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:08.623070 (+ 3164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:39635: BlockingRecv error: recv got EOF from 127.31.43.194:39635 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 12 03:58:08 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:08 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:08.771435 10015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:08.765520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:56809)
0812 03:58:08.765729 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:08.765741 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:08.765884 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:08.766132 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:08.766133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:08.766417 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:08.766498 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:08.766501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:08.767528 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:08.767529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:08.768140 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:08.768143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:08.768534 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:08.768535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:08.768581 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:08.769214 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:08.769220 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:08.771365 (+ 2145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":91,"threads_started":1}
W20260812 03:58:08.771610 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:08.765635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:56809 (local address 127.31.43.254:42917)
0812 03:58:08.765710 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:08.765713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:08.765979 (+ 266us) server_negotiation.cc:407] Connection header received
0812 03:58:08.766016 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:08.766017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:08.766030 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:08.766112 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:08.766570 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:08.767383 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:08.768248 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:08.768452 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:08.768521 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:08.771497 (+ 2976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:56809: BlockingRecv error: recv got EOF from 127.31.43.195:56809 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 03:58:09 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:09 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:09.487133 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:09.475755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:34901)
0812 03:58:09.475956 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:09.475972 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:09.476054 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:09.476277 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:09.476278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:09.476493 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:09.476549 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:09.476551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:09.477332 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:09.477333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:09.478124 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:09.478127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:09.478625 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:09.478626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:09.478738 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:09.484483 (+ 5745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:09.484489 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:09.487030 (+ 2541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":103,"threads_started":1}
W20260812 03:58:09.487308 10017 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:09.475880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:34901 (local address 127.31.43.254:42917)
0812 03:58:09.476028 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:09.476030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:09.476044 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:58:09.476134 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:09.476135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:09.476148 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:09.476223 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:09.476657 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:09.477238 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:09.478244 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:09.478478 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:09.478571 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:09.487214 (+ 8643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:34901: BlockingRecv error: recv got EOF from 127.31.43.193:34901 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
Aug 12 03:58:09 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:09 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:09.629092 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:09.623551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:38085)
0812 03:58:09.623753 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:09.623771 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:09.623886 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:09.624097 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:09.624098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:09.624280 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:09.624332 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:09.624334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:09.625043 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:09.625044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:09.625608 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:09.625610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:09.625985 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:09.625986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:09.626103 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:09.626579 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:09.626584 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:09.629027 (+ 2443us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":114,"threads_started":1}
W20260812 03:58:09.629243 10017 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:09.623637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:38085 (local address 127.31.43.254:42917)
0812 03:58:09.623733 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:09.623736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:09.623853 (+ 117us) server_negotiation.cc:407] Connection header received
0812 03:58:09.623976 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:09.623977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:09.623990 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:09.624066 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:09.624415 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:09.624948 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:09.625704 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:09.625908 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:09.625987 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:09.629151 (+ 3164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:38085: BlockingRecv error: recv got EOF from 127.31.43.194:38085 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 03:58:09 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:09 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:09.777606 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:09.772202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:35657)
0812 03:58:09.772503 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:09.772519 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:09.772613 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:09.772795 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:09.772796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:09.773066 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:09.773147 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:09.773150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:09.773877 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:09.773878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:09.774424 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:09.774426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:09.774887 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:09.774888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:09.774967 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:09.775448 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:09.775454 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:09.777502 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":128,"threads_started":1}
W20260812 03:58:09.777661 10017 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:09.772346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:35657 (local address 127.31.43.254:42917)
0812 03:58:09.772468 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:09.772472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:09.772588 (+ 116us) server_negotiation.cc:407] Connection header received
0812 03:58:09.772696 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:09.772697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:09.772712 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:09.772787 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:09.773239 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:09.773788 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:09.774524 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:09.774785 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:09.774884 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:09.777594 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:35657: BlockingRecv error: recv got EOF from 127.31.43.195:35657 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 03:58:10 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:10 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:10.493242 10021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:10.487752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:52457)
0812 03:58:10.487922 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:10.487939 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:10.488007 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:10.488306 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:10.488307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:10.488490 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:10.488546 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:10.488548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:10.489312 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:10.489313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:10.489861 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:10.489862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:10.490482 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:10.490483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:10.490567 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:10.491165 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:10.491170 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:10.493177 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":104,"threads_started":1}
W20260812 03:58:10.493456 10022 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:10.487900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:52457 (local address 127.31.43.254:42917)
0812 03:58:10.488131 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:10.488134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:10.488150 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:58:10.488179 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:10.488179 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:10.488190 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:10.488254 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:10.488640 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:10.489218 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:10.489962 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:10.490371 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:10.490447 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:10.493371 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:52457: BlockingRecv error: recv got EOF from 127.31.43.193:52457 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":162,"threads_started":1}
Aug 12 03:58:10 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:10 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:10.634902 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:10.629734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:42015)
0812 03:58:10.629895 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:10.629907 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:10.629981 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:10.630173 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:10.630174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:10.630353 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:10.630408 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:10.630410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:10.631182 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:10.631183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:10.631714 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:10.631715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:10.632131 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:10.632131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:10.632183 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:10.632670 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:10.632675 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:10.634823 (+ 2148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
W20260812 03:58:10.635046 10022 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:10.629829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:42015 (local address 127.31.43.254:42917)
0812 03:58:10.629930 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:10.629932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:10.629969 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:58:10.630068 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:10.630069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:10.630081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:10.630152 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:10.630476 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:10.631090 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:10.631842 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:10.632050 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:10.632115 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:10.634972 (+ 2857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:42015: BlockingRecv error: recv got EOF from 127.31.43.194:42015 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 12 03:58:10 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:10 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:10.783877 10025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:10.778306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:41589)
0812 03:58:10.778526 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:10.778539 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:10.778626 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:10.778836 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:10.778837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:10.779089 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:10.779145 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:10.779148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:10.779823 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:10.779824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:10.780439 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:10.780442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:10.780880 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:10.780882 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:10.780966 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:10.781650 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:10.781656 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:10.783814 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":117,"threads_started":1}
W20260812 03:58:10.783975 10022 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:10.778459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:41589 (local address 127.31.43.254:42917)
0812 03:58:10.778541 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:10.778543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:10.778623 (+ 80us) server_negotiation.cc:407] Connection header received
0812 03:58:10.778725 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:10.778726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:10.778747 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:10.778799 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:10.779226 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:10.779734 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:10.780519 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:10.780730 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:10.780794 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:10.783924 (+ 3130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:41589: BlockingRecv error: recv got EOF from 127.31.43.195:41589 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 03:58:11 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:11 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:11.499964 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:11.493874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:34963)
0812 03:58:11.494051 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:11.494064 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:11.494137 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:11.494632 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:11.494633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:11.494927 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:11.495005 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:11.495008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:11.495982 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:11.495983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:11.496548 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:11.496550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:11.496945 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:11.496946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:11.496993 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:11.497562 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:11.497567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:11.499891 (+ 2324us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":112,"threads_started":1}
W20260812 03:58:11.500216 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:11.494187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:34963 (local address 127.31.43.254:42917)
0812 03:58:11.494404 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:11.494407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:11.494420 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:58:11.494473 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:11.494474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:11.494506 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:11.494608 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:11.495151 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:11.495895 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:11.496641 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:11.496867 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:11.496937 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:11.500130 (+ 3193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:34963: BlockingRecv error: recv got EOF from 127.31.43.193:34963 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":82,"threads_started":1}
Aug 12 03:58:11 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:11 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:11.640604 10028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:11.635430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:56911)
0812 03:58:11.635632 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:11.635644 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:11.635708 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:11.635840 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:11.635841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:11.636011 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:11.636085 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:11.636087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:11.636926 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:11.636927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:11.637545 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:11.637548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:11.637927 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:11.637927 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:11.637969 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:11.638510 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:11.638515 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:11.640515 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":93,"threads_started":1}
W20260812 03:58:11.640781 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:11.635497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:56911 (local address 127.31.43.254:42917)
0812 03:58:11.635570 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:11.635572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:11.635706 (+ 134us) server_negotiation.cc:407] Connection header received
0812 03:58:11.635755 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:11.635756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:11.635767 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:11.635817 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:11.636152 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:11.636852 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:11.637616 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:11.637857 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:11.637912 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:11.640663 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:56911: BlockingRecv error: recv got EOF from 127.31.43.194:56911 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 03:58:11 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:11 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:11.790357 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:11.784606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:33855)
0812 03:58:11.784827 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:11.784841 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:11.784931 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:11.785175 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:11.785177 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:11.785389 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:11.785473 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:11.785475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:11.786208 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:11.786210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:11.786815 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:11.786818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:11.787231 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:11.787232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:11.787299 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:11.787825 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:11.787830 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:11.790295 (+ 2465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":103,"threads_started":1}
W20260812 03:58:11.790617 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:11.784744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:33855 (local address 127.31.43.254:42917)
0812 03:58:11.784832 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:11.784834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:11.784905 (+ 71us) server_negotiation.cc:407] Connection header received
0812 03:58:11.785054 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:11.785054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:11.785067 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:11.785163 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:11.785580 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:11.786099 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:11.786917 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:11.787133 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:11.787217 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:11.790542 (+ 3325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:33855: BlockingRecv error: recv got EOF from 127.31.43.195:33855 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 03:58:12 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:12 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:12.507061 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:12.500615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:49189)
0812 03:58:12.500794 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:12.500824 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:12.500908 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:12.501238 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:12.501239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:12.501479 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:12.501550 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:12.501552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:12.502420 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:12.502421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:12.503177 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:12.503181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:12.503714 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:12.503715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:12.503760 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:12.504386 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:12.504391 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:12.506940 (+ 2549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
W20260812 03:58:12.507220 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:12.500865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:49189 (local address 127.31.43.254:42917)
0812 03:58:12.501016 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:12.501019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:12.501032 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:58:12.501068 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:12.501069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:12.501081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:12.501164 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:12.501651 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:12.502309 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:12.503274 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:12.503625 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:12.503712 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:12.507134 (+ 3422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:49189: BlockingRecv error: recv got EOF from 127.31.43.193:49189 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
Aug 12 03:58:12 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:12 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:12.646183 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:12.641155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:59537)
0812 03:58:12.641305 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:12.641316 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:12.641378 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:12.641539 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:12.641539 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:12.641733 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:12.641785 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:12.641787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:12.642485 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:12.642486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:12.643132 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:12.643136 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:12.643568 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:12.643569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:12.643615 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:12.644206 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:12.644211 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:12.646117 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
W20260812 03:58:12.646340 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:12.641249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:59537 (local address 127.31.43.254:42917)
0812 03:58:12.641306 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:12.641308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:12.641361 (+ 53us) server_negotiation.cc:407] Connection header received
0812 03:58:12.641426 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:12.641427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:12.641437 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:12.641495 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:12.641870 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:12.642393 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:12.643276 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:12.643474 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:12.643551 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:12.646247 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:59537: BlockingRecv error: recv got EOF from 127.31.43.194:59537 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 03:58:12 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:12 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:12.796756 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:12.790996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:38553)
0812 03:58:12.791215 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:12.791232 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:12.791309 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:12.791486 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:12.791487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:12.791771 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:12.791828 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:12.791830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:12.792641 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:12.792641 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:12.793321 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:12.793323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:12.793727 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:12.793729 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:12.793783 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:12.794387 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:12.794392 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:12.796690 (+ 2298us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":155,"threads_started":1}
W20260812 03:58:12.796888 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:12.791170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:38553 (local address 127.31.43.254:42917)
0812 03:58:12.791243 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:12.791250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:12.791308 (+ 58us) server_negotiation.cc:407] Connection header received
0812 03:58:12.791376 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:12.791377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:12.791388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:12.791459 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:12.791938 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:12.792533 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:12.793419 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:12.793619 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:12.793681 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:12.796801 (+ 3120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:38553: BlockingRecv error: recv got EOF from 127.31.43.195:38553 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 03:58:13 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:13 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:13.512881 10034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:13.507760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:36027)
0812 03:58:13.507938 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:13.507950 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:13.508019 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:13.508168 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:13.508169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:13.508359 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:13.508421 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:13.508423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:13.509093 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:13.509094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:13.509657 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:13.509659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:13.510042 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:13.510043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:13.510085 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:13.510714 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:13.510719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:13.512776 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
W20260812 03:58:13.512949 10035 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:13.507884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:36027 (local address 127.31.43.254:42917)
0812 03:58:13.508034 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:13.508037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:13.508045 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:58:13.508073 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:13.508073 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:13.508084 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:13.508144 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:13.508501 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:13.509012 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:13.509763 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:13.509967 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:13.510023 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:13.512874 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:36027: BlockingRecv error: recv got EOF from 127.31.43.193:36027 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
Aug 12 03:58:13 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:13 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:13.652755 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:13.646916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:44425)
0812 03:58:13.647068 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:13.647086 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:13.647178 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:13.647745 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:13.647746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:13.647969 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:13.648037 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:13.648039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:13.648840 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:13.648842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:13.649447 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:13.649449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:13.649883 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:13.649885 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:13.649953 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:13.650430 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:13.650436 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:13.652677 (+ 2241us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
W20260812 03:58:13.653012 10035 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:13.647507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:44425 (local address 127.31.43.254:42917)
0812 03:58:13.647571 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:13.647573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:13.647587 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:58:13.647626 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:13.647627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:13.647641 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:13.647711 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:13.648158 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:13.648724 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:13.649556 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:13.649785 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:13.649890 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:13.652926 (+ 3036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:44425: BlockingRecv error: recv got EOF from 127.31.43.194:44425 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 03:58:13 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:13 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:13.803577 10037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:13.797514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:53725)
0812 03:58:13.797743 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:13.797756 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:13.797865 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:13.798074 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:13.798075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:13.798295 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:13.798350 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:13.798352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:13.799239 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:13.799241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:13.799857 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:13.799859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:13.800299 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:13.800301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:13.800372 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:13.801078 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:13.801084 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:13.803487 (+ 2403us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":95,"threads_started":1}
W20260812 03:58:13.803686 10035 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:13.797769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:53725 (local address 127.31.43.254:42917)
0812 03:58:13.797862 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:13.797865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:13.797878 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:58:13.797939 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:13.797940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:13.797964 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:13.798071 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:13.798461 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:13.799113 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:13.799980 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:13.800201 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:13.800298 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:13.803618 (+ 3320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:53725: BlockingRecv error: recv got EOF from 127.31.43.195:53725 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 03:58:14 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:14 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:14.524073 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:14.513522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:57901)
0812 03:58:14.513705 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:14.513724 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:14.513782 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:14.513971 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:14.513971 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:14.514188 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:14.514242 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:14.514244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:14.515144 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:14.515145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:14.515703 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:14.515705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:14.516162 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:14.516163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:14.516218 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:14.521804 (+ 5586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:14.521810 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:14.523993 (+ 2183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":117,"threads_started":1}
W20260812 03:58:14.524302 10039 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:14.513659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:57901 (local address 127.31.43.254:42917)
0812 03:58:14.513817 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:14.513819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:14.513829 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:58:14.513858 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:14.513859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:14.513870 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:14.513952 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:14.514363 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:14.515021 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:14.515824 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:14.516054 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:14.516125 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:14.524201 (+ 8076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:57901: BlockingRecv error: recv got EOF from 127.31.43.193:57901 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
Aug 12 03:58:14 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:14 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:14.658941 10040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:14.653507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:43279)
0812 03:58:14.653708 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:14.653720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:14.653787 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:14.654027 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:14.654028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:14.654275 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:14.654348 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:14.654350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:14.655319 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:14.655320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:14.655862 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:14.655864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:14.656318 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:14.656319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:14.656364 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:14.656842 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:14.656847 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:14.658852 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":111,"threads_started":1}
W20260812 03:58:14.659114 10039 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:14.653598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:43279 (local address 127.31.43.254:42917)
0812 03:58:14.653684 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:14.653691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:14.653770 (+ 79us) server_negotiation.cc:407] Connection header received
0812 03:58:14.653907 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:14.653908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:14.653922 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:14.654019 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:14.654434 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:14.655250 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:14.656018 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:14.656214 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:14.656286 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:14.659003 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:43279: BlockingRecv error: recv got EOF from 127.31.43.194:43279 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 03:58:14 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:14 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:14.810196 10041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:14.804319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:60251)
0812 03:58:14.804571 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:14.804584 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:14.804665 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:14.804929 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:14.804931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:14.805121 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:14.805175 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:14.805177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:14.805968 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:14.805969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:14.806506 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:14.806508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:14.807036 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:14.807037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:14.807100 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:14.807630 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:14.807636 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:14.810107 (+ 2471us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":220,"threads_started":1}
W20260812 03:58:14.810266 10039 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:14.804495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:60251 (local address 127.31.43.254:42917)
0812 03:58:14.804571 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:14.804574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:14.804661 (+ 87us) server_negotiation.cc:407] Connection header received
0812 03:58:14.804721 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:14.804721 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:14.804734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:14.804800 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:14.805300 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:14.805833 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:14.806604 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:14.806838 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:14.806906 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:14.810216 (+ 3310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:60251: BlockingRecv error: recv got EOF from 127.31.43.195:60251 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 03:58:15 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:15 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:15.530416 10042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:15.524716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:54129)
0812 03:58:15.524900 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:15.524920 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:15.524990 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:15.525284 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:15.525286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:15.525492 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:15.525550 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:15.525551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:15.526849 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:15.526851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:15.527392 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:15.527395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:15.527775 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:15.527776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:15.527820 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:15.528323 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:15.528328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:15.530350 (+ 2022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":123,"threads_started":1}
W20260812 03:58:15.530709 10043 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:15.524903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:54129 (local address 127.31.43.254:42917)
0812 03:58:15.525100 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:15.525102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:15.525111 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:58:15.525141 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:15.525142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:15.525153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:15.525227 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:15.525710 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:15.526610 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:15.527485 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:15.527696 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:15.527757 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:15.530492 (+ 2735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:54129: BlockingRecv error: recv got EOF from 127.31.43.193:54129 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":69,"threads_started":1}
Aug 12 03:58:15 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:15 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:15.664709 10044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:15.659583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:49907)
0812 03:58:15.659750 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:15.659768 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:15.659831 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:15.659995 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:15.659996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:15.660262 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:15.660326 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:15.660328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:15.661133 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:15.661134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:15.661681 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:15.661683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:15.662086 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:15.662087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:15.662152 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:15.662635 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:15.662640 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:15.664645 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
W20260812 03:58:15.664870 10043 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:15.659677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:49907 (local address 127.31.43.254:42917)
0812 03:58:15.659732 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:15.659735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:15.659833 (+ 98us) server_negotiation.cc:407] Connection header received
0812 03:58:15.659891 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:15.659892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:15.659903 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:15.659968 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:15.660418 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:15.661030 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:15.661782 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:15.661991 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:15.662084 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:15.664794 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:49907: BlockingRecv error: recv got EOF from 127.31.43.194:49907 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 03:58:15 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:15 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:15.816733 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:15.810872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:58667)
0812 03:58:15.811117 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:15.811131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:15.811213 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:15.811371 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:15.811372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:15.811594 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:15.811663 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:15.811665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:15.812372 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:15.812373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:15.813004 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:15.813007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:15.813616 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:15.813617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:15.813684 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:15.814333 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:15.814339 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:15.816666 (+ 2327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":178,"threads_started":1}
W20260812 03:58:15.816927 10043 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:15.811068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:58667 (local address 127.31.43.254:42917)
0812 03:58:15.811121 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:15.811123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:15.811198 (+ 75us) server_negotiation.cc:407] Connection header received
0812 03:58:15.811266 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:15.811267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:15.811280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:15.811340 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:15.811746 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:15.812272 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:15.813303 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:15.813510 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:15.813579 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:15.816846 (+ 3267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:58667: BlockingRecv error: recv got EOF from 127.31.43.195:58667 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 12 03:58:16 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:16 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:16.536580 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:16.531051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:43403)
0812 03:58:16.531195 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:16.531214 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:16.531269 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:16.531530 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:16.531531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:16.531714 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:16.531762 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:16.531763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:16.532535 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:16.532536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:16.533140 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:16.533143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:16.533536 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:16.533537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:16.533648 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:16.534274 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:16.534279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:16.536494 (+ 2215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
W20260812 03:58:16.536721 10048 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:16.531201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:43403 (local address 127.31.43.254:42917)
0812 03:58:16.531359 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:16.531361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:16.531370 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:58:16.531396 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:16.531397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:16.531408 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:16.531479 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:16.531865 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:16.532443 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:16.533235 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:16.533457 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:16.533525 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:16.536622 (+ 3097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:43403: BlockingRecv error: recv got EOF from 127.31.43.193:43403 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
Aug 12 03:58:16 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:16 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:16.670742 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:16.665254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:44577)
0812 03:58:16.665529 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:16.665541 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:16.665622 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:16.665785 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:16.665786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:16.665985 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:16.666045 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:16.666048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:16.666806 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:16.666807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:16.667399 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:16.667402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:16.667780 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:16.667781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:16.667829 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:16.668561 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:16.668567 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:16.670607 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":156,"threads_started":1}
W20260812 03:58:16.670933 10048 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:16.665426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:44577 (local address 127.31.43.254:42917)
0812 03:58:16.665505 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:16.665507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:16.665615 (+ 108us) server_negotiation.cc:407] Connection header received
0812 03:58:16.665688 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:16.665688 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:16.665708 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:16.665774 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:16.666139 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:16.666655 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:16.667488 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:16.667691 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:16.667752 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:16.670843 (+ 3091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:44577: BlockingRecv error: recv got EOF from 127.31.43.194:44577 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 03:58:16 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:16 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:16.823122 10050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:16.817466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:43117)
0812 03:58:16.817655 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:16.817667 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:16.817750 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:16.817923 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:16.817923 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:16.818133 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:16.818191 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:16.818192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:16.819100 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:16.819102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:16.819844 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:16.819847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:16.820311 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:16.820312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:16.820356 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:16.820845 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:16.820850 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:16.823039 (+ 2189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":125,"threads_started":1}
W20260812 03:58:16.823262 10048 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:16.817576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:43117 (local address 127.31.43.254:42917)
0812 03:58:16.817681 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:16.817683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:16.817748 (+ 65us) server_negotiation.cc:407] Connection header received
0812 03:58:16.817829 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:16.817830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:16.817848 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:16.817896 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:16.818316 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:16.819008 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:16.819960 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:16.820231 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:16.820298 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:16.823187 (+ 2889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:43117: BlockingRecv error: recv got EOF from 127.31.43.195:43117 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 12 03:58:17 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:17 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:17.542780 10051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:17.537367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:33755)
0812 03:58:17.537548 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:17.537567 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:17.537638 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:17.537783 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:17.537783 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:17.537962 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:17.538023 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:17.538026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:17.538797 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:17.538798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:17.539363 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:17.539366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:17.539756 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:17.539756 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:17.539804 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:17.540296 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:17.540301 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:17.542641 (+ 2340us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
W20260812 03:58:17.542876 10052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:17.537466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:33755 (local address 127.31.43.254:42917)
0812 03:58:17.537634 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:17.537636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:17.537647 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:58:17.537692 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:17.537693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:17.537704 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:17.537764 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:17.538097 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:17.538696 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:17.539453 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:17.539678 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:17.539735 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:17.542794 (+ 3059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:33755: BlockingRecv error: recv got EOF from 127.31.43.193:33755 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
Aug 12 03:58:17 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:17 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:17.676971 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:17.671390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:38243)
0812 03:58:17.671615 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:17.671628 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:17.671730 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:17.671927 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:17.671929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:17.672237 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:17.672292 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:17.672295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:17.673141 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:17.673142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:17.673764 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:17.673766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:17.674196 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:17.674197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:17.674246 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:17.674959 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:17.674964 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:17.676865 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":129,"threads_started":1}
W20260812 03:58:17.677103 10052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:17.671495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:38243 (local address 127.31.43.254:42917)
0812 03:58:17.671550 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:17.671553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:17.671703 (+ 150us) server_negotiation.cc:407] Connection header received
0812 03:58:17.671802 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:17.671803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:17.671815 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:17.671870 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:17.672373 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:17.672939 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:17.673845 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:17.674103 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:17.674162 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:17.676999 (+ 2837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:38243: BlockingRecv error: recv got EOF from 127.31.43.194:38243 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 12 03:58:17 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:17 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.195@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:17.828773 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:17.823671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.195:49283)
0812 03:58:17.823873 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:17.823891 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:17.823960 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:17.824159 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:17.824160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:17.824348 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:17.824401 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:17.824403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:17.825150 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:17.825151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:17.825699 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:17.825700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:17.826093 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:17.826094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:17.826140 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:17.826705 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:17.826712 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:17.828700 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":106,"threads_started":1}
W20260812 03:58:17.828917 10052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:17.823765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:49283 (local address 127.31.43.254:42917)
0812 03:58:17.823832 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:17.823834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:17.823957 (+ 123us) server_negotiation.cc:407] Connection header received
0812 03:58:17.824010 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:17.824011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:17.824025 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:17.824077 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:17.824497 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:17.825068 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:17.825782 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:17.825969 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:17.826080 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:17.828826 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.195:49283: BlockingRecv error: recv got EOF from 127.31.43.195:49283 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 03:58:18 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:18 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.193@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:18.548590 10055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:18.543401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.193:52747)
0812 03:58:18.543583 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:18.543598 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:18.543677 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:18.543812 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:18.543813 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:18.544032 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:18.544086 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:18.544088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:18.544881 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:18.544881 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:18.545468 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:18.545471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:18.545839 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:18.545840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:18.545883 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:18.546497 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:18.546504 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:18.548513 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
W20260812 03:58:18.548800 10056 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:18.543460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:52747 (local address 127.31.43.254:42917)
0812 03:58:18.543617 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:18.543619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:18.543659 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:58:18.543723 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:18.543723 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:18.543735 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:18.543791 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:18.544219 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:18.544791 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:18.545557 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:18.545770 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:18.545828 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:18.548712 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.193:52747: BlockingRecv error: recv got EOF from 127.31.43.193:52747 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
Aug 12 03:58:18 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:58:18 dist-test-slave-8tdl krb5kdc[8861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.43.194@KRBTEST.COM for oryx/127.31.43.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:58:18.683120 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:18.677710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42917 (local address 127.31.43.194:47463)
0812 03:58:18.677905 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:18.677921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:18.677982 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:18.678133 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:18.678134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:18.678312 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:18.678364 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:18.678365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:18.679138 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:18.679139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:18.679678 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:18.679680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:18.680086 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:18.680087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:18.680163 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:18.680772 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:18.680786 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:18.682963 (+ 2177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.43.254:42917: Server oryx/127.31.43.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":95,"threads_started":1}
W20260812 03:58:18.683163 10056 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:18.677755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:47463 (local address 127.31.43.254:42917)
0812 03:58:18.677828 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:18.677830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:18.678004 (+ 174us) server_negotiation.cc:407] Connection header received
0812 03:58:18.678040 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:18.678040 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:18.678053 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:18.678110 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:18.678440 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:18.679050 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:18.679771 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:18.679973 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:18.680054 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:18.683096 (+ 3042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.43.194:47463: BlockingRecv error: recv got EOF from 127.31.43.194:47463 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:58:18.732082 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 9516
I20260812 03:58:18.742399 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 9684
I20260812 03:58:18.751569 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 9843
I20260812 03:58:18.760746 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 9440
2026-08-12T03:58:18Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17780 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T03:58:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:58:18Z Disabled control of system clock
I20260812 03:58:18.797794 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:35273
--webserver_interface=127.31.43.254
--webserver_port=0
--builtin_ntp_servers=127.31.43.212:45241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:35273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 03:58:18.901160 10062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:18.901439 10062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:18.901464 10062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:18.903247 10062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:58:18.903328 10062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:18.903347 10062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:18.903370 10062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:58:18.903391 10062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:58:18.905547 10062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:35273
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:35273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:18.905946 10062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:18.906368 10062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:18.913492 10068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:18.913523 10071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:18.913545 10062 server_base.cc:1034] running on GCE node
W20260812 03:58:18.913461 10069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:18.914348 10062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:18.914811 10062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:18.916028 10062 hybrid_clock.cc:648] HybridClock initialized: now 1786507098916004 us; error 53 us; skew 500 ppm
I20260812 03:58:18.918085 10062 webserver.cc:460] Webserver started at http://127.31.43.254:36515/ using document root <none> and password file <none>
I20260812 03:58:18.918562 10062 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:18.918664 10062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:18.919006 10062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:18.920804 10062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "e822bee7577b4eb789ac19cf4fb2750d"
format_stamp: "Formatted at 2026-08-12 03:58:18 on dist-test-slave-8tdl"
server_key: "51429d433f6c885e470fef48572fbd4d"
server_key_iv: "876aac3877ffe22a5924e6922431670d"
server_key_version: "encryptionkey@0"
I20260812 03:58:18.921324 10062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "e822bee7577b4eb789ac19cf4fb2750d"
format_stamp: "Formatted at 2026-08-12 03:58:18 on dist-test-slave-8tdl"
server_key: "51429d433f6c885e470fef48572fbd4d"
server_key_iv: "876aac3877ffe22a5924e6922431670d"
server_key_version: "encryptionkey@0"
I20260812 03:58:18.923797 10062 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.002s
I20260812 03:58:18.925526 10079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:18.925870 10062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:18.926133 10062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "e822bee7577b4eb789ac19cf4fb2750d"
format_stamp: "Formatted at 2026-08-12 03:58:18 on dist-test-slave-8tdl"
server_key: "51429d433f6c885e470fef48572fbd4d"
server_key_iv: "876aac3877ffe22a5924e6922431670d"
server_key_version: "encryptionkey@0"
I20260812 03:58:18.926337 10062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:18.948170 10062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:18.948705 10062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:18.948987 10062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:18.955973 10062 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:35273
I20260812 03:58:18.956092 10137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:35273 every 8 connection(s)
I20260812 03:58:18.956616 10062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:58:18.957762 10138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:58:18.962080 10138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d: Bootstrap starting.
I20260812 03:58:18.963109 10138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d: Neither blocks nor log segments found. Creating new log.
I20260812 03:58:18.963487 10138 log.cc:824] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:58:18.964427 10138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d: No bootstrap required, opened a new log
I20260812 03:58:18.964529 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 10062
I20260812 03:58:18.964705 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:58:18.964972 31919 external_mini_cluster.cc:1428] Setting key 7b68b7691546a2746d25c5627d059767
I20260812 03:58:18.967768 10138 raft_consensus.cc:348] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e822bee7577b4eb789ac19cf4fb2750d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35273 } }
I20260812 03:58:18.968327 10138 raft_consensus.cc:374] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:58:18.968427 10138 raft_consensus.cc:729] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e822bee7577b4eb789ac19cf4fb2750d, State: Initialized, Role: FOLLOWER
I20260812 03:58:18.968665 10138 consensus_queue.cc:260] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e822bee7577b4eb789ac19cf4fb2750d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35273 } }
I20260812 03:58:18.968812 10138 raft_consensus.cc:388] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:58:18.968886 10138 raft_consensus.cc:482] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:58:18.968942 10138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:58:18.970377 10138 raft_consensus.cc:504] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e822bee7577b4eb789ac19cf4fb2750d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35273 } }
I20260812 03:58:18.970585 10138 leader_election.cc:302] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e822bee7577b4eb789ac19cf4fb2750d; no voters:
I20260812 03:58:18.970875 10138 leader_election.cc:288] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:58:18.970975 10144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:58:18.971280 10144 raft_consensus.cc:686] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [term 1 LEADER]: Becoming Leader. State: Replica: e822bee7577b4eb789ac19cf4fb2750d, State: Running, Role: LEADER
I20260812 03:58:18.971302 10138 sys_catalog.cc:564] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:58:18.971444 10144 consensus_queue.cc:237] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e822bee7577b4eb789ac19cf4fb2750d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35273 } }
I20260812 03:58:18.972061 10144 sys_catalog.cc:455] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e822bee7577b4eb789ac19cf4fb2750d. Latest consensus state: current_term: 1 leader_uuid: "e822bee7577b4eb789ac19cf4fb2750d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e822bee7577b4eb789ac19cf4fb2750d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35273 } } }
I20260812 03:58:18.972087 10145 sys_catalog.cc:455] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e822bee7577b4eb789ac19cf4fb2750d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e822bee7577b4eb789ac19cf4fb2750d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35273 } } }
I20260812 03:58:18.972426 10144 sys_catalog.cc:458] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [sys.catalog]: This master's current role is: LEADER
I20260812 03:58:18.972441 10145 sys_catalog.cc:458] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d [sys.catalog]: This master's current role is: LEADER
I20260812 03:58:18.972864 10151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:58:18.973807 10151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:58:18.977439 10151 catalog_manager.cc:1347] Generated new cluster ID: 878f7a928f0040158f7e98ce3715ed1e
I20260812 03:58:18.977550 10151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:58:18.977797 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:18.965912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51526 (local address 127.31.43.254:35273)
0812 03:58:18.966203 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:18.966211 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:18.966248 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:58:18.967775 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:18.967785 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:18.967903 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:18.968155 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:18.968700 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:18.971513 (+ 2813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:18.974193 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:18.975345 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:18.976087 (+ 742us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:18.976685 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:18.976695 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:18.976695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:58:18.976912 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:18.976947 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:58:18.976960 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:18.977058 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:18.977182 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 03:58:18.977422 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":124,"threads_started":1}
I20260812 03:58:19.002722 10151 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:58:19.003873 10151 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:58:19.007143 10151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e822bee7577b4eb789ac19cf4fb2750d: Generated new TSK 0
I20260812 03:58:19.007416 10151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:58:19.018009 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:35273
--builtin_ntp_servers=127.31.43.212:45241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 03:58:19.147265 10167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:19.147591 10167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:19.147625 10167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:19.149385 10167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:19.149475 10167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:19.149509 10167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:58:19.151826 10167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:35273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:19.152235 10167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:19.152763 10167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:19.161881 10176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:19.161947 10178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:19.162292 10167 server_base.cc:1034] running on GCE node
W20260812 03:58:19.161881 10175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:19.162839 10167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:19.163226 10167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:19.164443 10167 hybrid_clock.cc:648] HybridClock initialized: now 1786507099164410 us; error 53 us; skew 500 ppm
I20260812 03:58:19.166465 10167 webserver.cc:460] Webserver started at http://127.31.43.193:35617/ using document root <none> and password file <none>
I20260812 03:58:19.166968 10167 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:19.167083 10167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:19.167425 10167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:19.169185 10167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "e6c3e66b2f1b445c97049712ce7a993d"
format_stamp: "Formatted at 2026-08-12 03:58:19 on dist-test-slave-8tdl"
server_key: "178dcb730108895f4f8134d272e89fad"
server_key_iv: "50cab35f49e8247a82d4f921ee05464d"
server_key_version: "encryptionkey@0"
I20260812 03:58:19.169806 10167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "e6c3e66b2f1b445c97049712ce7a993d"
format_stamp: "Formatted at 2026-08-12 03:58:19 on dist-test-slave-8tdl"
server_key: "178dcb730108895f4f8134d272e89fad"
server_key_iv: "50cab35f49e8247a82d4f921ee05464d"
server_key_version: "encryptionkey@0"
I20260812 03:58:19.172149 10167 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20260812 03:58:19.173388 10184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:19.173668 10167 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:19.173838 10167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "e6c3e66b2f1b445c97049712ce7a993d"
format_stamp: "Formatted at 2026-08-12 03:58:19 on dist-test-slave-8tdl"
server_key: "178dcb730108895f4f8134d272e89fad"
server_key_iv: "50cab35f49e8247a82d4f921ee05464d"
server_key_version: "encryptionkey@0"
I20260812 03:58:19.173997 10167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:19.203142 10167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:19.203683 10167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:19.203918 10167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:19.204244 10167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:19.204871 10167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:19.204947 10167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:19.205041 10167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:19.205125 10167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:19.215318 10167 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:38041
I20260812 03:58:19.215399 10313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:38041 every 8 connection(s)
I20260812 03:58:19.215955 10167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:58:19.218533 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 10167
I20260812 03:58:19.218667 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:58:19.218973 31919 external_mini_cluster.cc:1428] Setting key 3da7e1592b22a37565ab1ef858c2b587
I20260812 03:58:19.220597 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:35273
--builtin_ntp_servers=127.31.43.212:45241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 03:58:19.224702 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:19.217035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:58943 (local address 127.31.43.254:35273)
0812 03:58:19.217144 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:19.217146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:19.217441 (+ 295us) server_negotiation.cc:407] Connection header received
0812 03:58:19.218160 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:19.218161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:19.218188 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:19.218290 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:19.219460 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:19.220302 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:19.222021 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:19.222488 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:19.222592 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:19.224182 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:19.224183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:19.224184 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:58:19.224190 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:19.224217 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:58:19.224223 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:19.224281 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:19.224556 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 03:58:19.224616 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:58:19.225409 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:19.217006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35273 (local address 127.31.43.193:58943)
0812 03:58:19.217288 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:19.217337 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:19.218001 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:19.218412 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:19.218416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:19.218426 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:19.219331 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:19.219338 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:19.220493 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:19.220495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:19.221896 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:19.221899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:19.222618 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:19.222619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:19.222910 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:19.223928 (+ 1018us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:58:19.223929 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:19.223987 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:58:19.224029 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:58:19.224306 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:58:19.224323 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:58:19.224327 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:58:19.224547 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 03:58:19.224805 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":140,"threads_started":1}
I20260812 03:58:19.226444 10314 heartbeater.cc:347] Connected to a master server at 127.31.43.254:35273
I20260812 03:58:19.226576 10314 heartbeater.cc:464] Registering TS with master...
I20260812 03:58:19.226970 10314 heartbeater.cc:511] Master 127.31.43.254:35273 requested a full tablet report, sending...
I20260812 03:58:19.227761 10097 ts_manager.cc:194] Registered new tserver with Master: e6c3e66b2f1b445c97049712ce7a993d (127.31.43.193:38041)
I20260812 03:58:19.228626 10097 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.43.193:58943
W20260812 03:58:19.362442 10317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:19.362897 10317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:19.362968 10317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:19.365068 10317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:19.365208 10317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:19.365315 10317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:58:19.368050 10317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:35273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:19.368595 10317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:19.369184 10317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:19.378567 10326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:19.378654 10324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:19.378608 10323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:19.379465 10317 server_base.cc:1034] running on GCE node
I20260812 03:58:19.379928 10317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:19.380440 10317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:19.381611 10317 hybrid_clock.cc:648] HybridClock initialized: now 1786507099381591 us; error 52 us; skew 500 ppm
I20260812 03:58:19.383783 10317 webserver.cc:460] Webserver started at http://127.31.43.194:38169/ using document root <none> and password file <none>
I20260812 03:58:19.384184 10317 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:19.384277 10317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:19.384562 10317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:19.386410 10317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "f92b010490e242a8862a019d2815e8c8"
format_stamp: "Formatted at 2026-08-12 03:58:19 on dist-test-slave-8tdl"
server_key: "97cb3383c2d082a2f54549047e2907fd"
server_key_iv: "4b689bf6f0063934bd08b350e2a059ab"
server_key_version: "encryptionkey@0"
I20260812 03:58:19.387125 10317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "f92b010490e242a8862a019d2815e8c8"
format_stamp: "Formatted at 2026-08-12 03:58:19 on dist-test-slave-8tdl"
server_key: "97cb3383c2d082a2f54549047e2907fd"
server_key_iv: "4b689bf6f0063934bd08b350e2a059ab"
server_key_version: "encryptionkey@0"
I20260812 03:58:19.389603 10317 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:58:19.391185 10333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:19.391502 10317 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:19.391719 10317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "f92b010490e242a8862a019d2815e8c8"
format_stamp: "Formatted at 2026-08-12 03:58:19 on dist-test-slave-8tdl"
server_key: "97cb3383c2d082a2f54549047e2907fd"
server_key_iv: "4b689bf6f0063934bd08b350e2a059ab"
server_key_version: "encryptionkey@0"
I20260812 03:58:19.391857 10317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:19.417663 10317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:19.418320 10317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:19.418598 10317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:19.419265 10317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:19.419895 10317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:19.419955 10317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:19.419992 10317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:19.420019 10317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:19.432245 10317 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:39369
I20260812 03:58:19.432379 10461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:39369 every 8 connection(s)
I20260812 03:58:19.433660 10317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:58:19.443030 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 10317
I20260812 03:58:19.443222 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:58:19.443504 31919 external_mini_cluster.cc:1428] Setting key bde119a9e8faa888df6f632e54032dd7
I20260812 03:58:19.445250 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:35273
--builtin_ntp_servers=127.31.43.212:45241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 03:58:19.445746 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:19.435302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:40545 (local address 127.31.43.254:35273)
0812 03:58:19.435441 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:19.435454 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:19.436255 (+ 801us) server_negotiation.cc:407] Connection header received
0812 03:58:19.437506 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:19.437508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:19.437530 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:19.437634 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:19.439669 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:19.440773 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:19.442938 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:19.443395 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:19.443516 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:19.445107 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:19.445109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:19.445109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:58:19.445126 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:19.445156 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:58:19.445163 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:19.445272 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:19.445560 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 03:58:19.445638 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:58:19.446539 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:19.435396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35273 (local address 127.31.43.194:40545)
0812 03:58:19.436005 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:19.436102 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:19.437310 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:19.437832 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:19.437845 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:19.437866 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:19.439424 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:19.439444 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:19.441058 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:19.441059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:19.442795 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:19.442799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:19.443554 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:19.443556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:19.443815 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:19.444831 (+ 1016us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:58:19.444831 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:19.444911 (+ 80us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:58:19.444968 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:58:19.445270 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:58:19.445275 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:58:19.445288 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 03:58:19.445995 (+ 707us) client_negotiation.cc:240] Negotiation successful
0812 03:58:19.446230 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":195,"threads_started":1}
I20260812 03:58:19.447633 10462 heartbeater.cc:347] Connected to a master server at 127.31.43.254:35273
I20260812 03:58:19.447932 10462 heartbeater.cc:464] Registering TS with master...
I20260812 03:58:19.448351 10462 heartbeater.cc:511] Master 127.31.43.254:35273 requested a full tablet report, sending...
I20260812 03:58:19.449278 10097 ts_manager.cc:194] Registered new tserver with Master: f92b010490e242a8862a019d2815e8c8 (127.31.43.194:39369)
I20260812 03:58:19.450268 10097 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.43.194:40545
W20260812 03:58:19.593847 10465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:19.594182 10465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:19.594218 10465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:19.596012 10465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:19.596092 10465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:19.596141 10465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:58:19.598404 10465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:35273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:19.598824 10465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:19.599350 10465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:19.607860 10471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:19.607867 10474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:19.607859 10472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:19.608479 10465 server_base.cc:1034] running on GCE node
I20260812 03:58:19.608853 10465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:19.609349 10465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:19.610582 10465 hybrid_clock.cc:648] HybridClock initialized: now 1786507099610569 us; error 66 us; skew 500 ppm
I20260812 03:58:19.617831 10465 webserver.cc:460] Webserver started at http://127.31.43.195:42671/ using document root <none> and password file <none>
I20260812 03:58:19.618221 10465 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:19.618346 10465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:19.618605 10465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:19.620400 10465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "cce44e0166d745e0bb96ee52bfdf3f75"
format_stamp: "Formatted at 2026-08-12 03:58:19 on dist-test-slave-8tdl"
server_key: "e774fccea8a0ff13dc753391e849a84e"
server_key_iv: "e143ccbc2f156e0316d17e778e109270"
server_key_version: "encryptionkey@0"
I20260812 03:58:19.621029 10465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "cce44e0166d745e0bb96ee52bfdf3f75"
format_stamp: "Formatted at 2026-08-12 03:58:19 on dist-test-slave-8tdl"
server_key: "e774fccea8a0ff13dc753391e849a84e"
server_key_iv: "e143ccbc2f156e0316d17e778e109270"
server_key_version: "encryptionkey@0"
I20260812 03:58:19.623613 10465 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 03:58:19.625017 10480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:19.625320 10465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:19.625473 10465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "cce44e0166d745e0bb96ee52bfdf3f75"
format_stamp: "Formatted at 2026-08-12 03:58:19 on dist-test-slave-8tdl"
server_key: "e774fccea8a0ff13dc753391e849a84e"
server_key_iv: "e143ccbc2f156e0316d17e778e109270"
server_key_version: "encryptionkey@0"
I20260812 03:58:19.625599 10465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:19.657781 10465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:19.658403 10465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:19.658725 10465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:19.659256 10465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:19.659919 10465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:19.660012 10465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:19.660058 10465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:19.660094 10465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:19.671556 10465 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:42631
I20260812 03:58:19.671630 10608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:42631 every 8 connection(s)
I20260812 03:58:19.672292 10465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:58:19.678365 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 10465
I20260812 03:58:19.678539 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:58:19.678864 31919 external_mini_cluster.cc:1428] Setting key cd5ed6e4828ad539f65f19bbc2638264
I20260812 03:58:19.681692 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:19.673664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:43741 (local address 127.31.43.254:35273)
0812 03:58:19.673749 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:19.673751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:19.674231 (+ 480us) server_negotiation.cc:407] Connection header received
0812 03:58:19.675297 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:19.675300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:19.675329 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:19.675419 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:19.676507 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:19.677197 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:19.679176 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:19.679508 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:19.679675 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:19.681278 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:19.681283 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:19.681284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:58:19.681290 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:19.681311 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:58:19.681321 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:19.681395 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:19.681556 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 03:58:19.681604 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:58:19.682199 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:19.673703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35273 (local address 127.31.43.195:43741)
0812 03:58:19.674057 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:19.674110 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:19.675121 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:19.675542 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:19.675547 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:19.675557 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:19.676354 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:19.676364 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:19.677356 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:19.677357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:19.679009 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:19.679018 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:19.679799 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:19.679800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:19.680002 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:19.681048 (+ 1046us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:58:19.681049 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:19.681102 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:58:19.681140 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:58:19.681428 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:58:19.681438 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:58:19.681445 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 03:58:19.681566 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 03:58:19.681723 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":122,"threads_started":1}
I20260812 03:58:19.683074 10609 heartbeater.cc:347] Connected to a master server at 127.31.43.254:35273
I20260812 03:58:19.683210 10609 heartbeater.cc:464] Registering TS with master...
I20260812 03:58:19.683549 10609 heartbeater.cc:511] Master 127.31.43.254:35273 requested a full tablet report, sending...
I20260812 03:58:19.684243 10097 ts_manager.cc:194] Registered new tserver with Master: cce44e0166d745e0bb96ee52bfdf3f75 (127.31.43.195:42631)
I20260812 03:58:19.684798 10097 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.43.195:43741
I20260812 03:58:19.692055 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:58:19.695154 10143 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:19.694181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51528 (local address 127.31.43.254:35273)
0812 03:58:19.694283 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:19.694285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:19.694471 (+ 186us) server_negotiation.cc:407] Connection header received
0812 03:58:19.694571 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:19.694572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:19.694592 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:19.694732 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:19.694981 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51528: BlockingRecv error: recv got EOF from 127.0.0.1:51528 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:58:19.695205 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 10167
I20260812 03:58:19.713027 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 10317
I20260812 03:58:19.728579 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 10465
I20260812 03:58:19.744184 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 10062
2026-08-12T03:58:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (990 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T03:58:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:58:19Z Disabled control of system clock
I20260812 03:58:19.792840 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:41181
--webserver_interface=127.31.43.254
--webserver_port=0
--builtin_ntp_servers=127.31.43.212:45539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:41181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:58:19.925873 10621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:19.926231 10621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:19.926275 10621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:19.928218 10621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:58:19.928328 10621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:19.928349 10621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:19.928367 10621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:58:19.928393 10621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:58:19.930640 10621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:41181
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:41181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:19.931232 10621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:19.931785 10621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:19.939337 10631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:19.939407 10629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:19.939337 10628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:19.939626 10621 server_base.cc:1034] running on GCE node
I20260812 03:58:19.940126 10621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:19.940615 10621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:19.941812 10621 hybrid_clock.cc:648] HybridClock initialized: now 1786507099941746 us; error 83 us; skew 500 ppm
I20260812 03:58:19.944022 10621 webserver.cc:460] Webserver started at http://127.31.43.254:39431/ using document root <none> and password file <none>
I20260812 03:58:19.944432 10621 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:19.944567 10621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:19.944778 10621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:19.946322 10621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "e401c743e7a24ee0a5084fd4a15710c1"
format_stamp: "Formatted at 2026-08-12 03:58:19 on dist-test-slave-8tdl"
server_key: "aaa9d004ae0a625e09f4071f3e982ab9"
server_key_iv: "39a417b3db666c4638a3a135616ced0e"
server_key_version: "encryptionkey@0"
I20260812 03:58:19.946956 10621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "e401c743e7a24ee0a5084fd4a15710c1"
format_stamp: "Formatted at 2026-08-12 03:58:19 on dist-test-slave-8tdl"
server_key: "aaa9d004ae0a625e09f4071f3e982ab9"
server_key_iv: "39a417b3db666c4638a3a135616ced0e"
server_key_version: "encryptionkey@0"
I20260812 03:58:19.949221 10621 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:58:19.950591 10637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:19.951097 10621 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:58:19.951227 10621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "e401c743e7a24ee0a5084fd4a15710c1"
format_stamp: "Formatted at 2026-08-12 03:58:19 on dist-test-slave-8tdl"
server_key: "aaa9d004ae0a625e09f4071f3e982ab9"
server_key_iv: "39a417b3db666c4638a3a135616ced0e"
server_key_version: "encryptionkey@0"
I20260812 03:58:19.951365 10621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:19.955590 10621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:19.956177 10621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:19.956395 10621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:19.962920 10621 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:41181
I20260812 03:58:19.963004 10696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:41181 every 8 connection(s)
I20260812 03:58:19.963555 10621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:58:19.964745 10697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:58:19.969195 10697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1: Bootstrap starting.
I20260812 03:58:19.970181 10697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1: Neither blocks nor log segments found. Creating new log.
I20260812 03:58:19.970652 10697 log.cc:824] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:58:19.972187 10697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1: No bootstrap required, opened a new log
I20260812 03:58:19.972178 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 10621
I20260812 03:58:19.972343 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:58:19.972702 31919 external_mini_cluster.cc:1428] Setting key 8083fa2e8420487423de2d3514b20093
I20260812 03:58:19.976048 10697 raft_consensus.cc:348] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e401c743e7a24ee0a5084fd4a15710c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41181 } }
I20260812 03:58:19.976403 10697 raft_consensus.cc:374] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:58:19.976509 10697 raft_consensus.cc:729] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e401c743e7a24ee0a5084fd4a15710c1, State: Initialized, Role: FOLLOWER
I20260812 03:58:19.976835 10697 consensus_queue.cc:260] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e401c743e7a24ee0a5084fd4a15710c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41181 } }
I20260812 03:58:19.977020 10697 raft_consensus.cc:388] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:58:19.977113 10697 raft_consensus.cc:482] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:58:19.977226 10697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:58:19.978331 10701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:19.973710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46348 (local address 127.31.43.254:41181)
0812 03:58:19.974041 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:19.974055 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:19.974101 (+ 46us) server_negotiation.cc:407] Connection header received
0812 03:58:19.975987 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:19.975996 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:19.976146 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:19.976368 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:19.977509 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:19.977513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:19.977513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:58:19.977518 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:19.977592 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:58:19.977613 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:19.977750 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:19.977894 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 03:58:19.978082 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":134,"threads_started":1}
I20260812 03:58:19.978502 10697 raft_consensus.cc:504] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e401c743e7a24ee0a5084fd4a15710c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41181 } }
I20260812 03:58:19.978705 10697 leader_election.cc:302] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e401c743e7a24ee0a5084fd4a15710c1; no voters:
I20260812 03:58:19.979044 10697 leader_election.cc:288] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:58:19.979588 10697 sys_catalog.cc:564] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:58:19.979799 10702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:58:19.980048 10702 raft_consensus.cc:686] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [term 1 LEADER]: Becoming Leader. State: Replica: e401c743e7a24ee0a5084fd4a15710c1, State: Running, Role: LEADER
I20260812 03:58:19.980224 10702 consensus_queue.cc:237] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e401c743e7a24ee0a5084fd4a15710c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41181 } }
I20260812 03:58:19.981031 10705 sys_catalog.cc:455] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e401c743e7a24ee0a5084fd4a15710c1. Latest consensus state: current_term: 1 leader_uuid: "e401c743e7a24ee0a5084fd4a15710c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e401c743e7a24ee0a5084fd4a15710c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41181 } } }
I20260812 03:58:19.981189 10705 sys_catalog.cc:458] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:58:19.981273 10703 sys_catalog.cc:455] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e401c743e7a24ee0a5084fd4a15710c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e401c743e7a24ee0a5084fd4a15710c1" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41181 } } }
I20260812 03:58:19.981467 10703 sys_catalog.cc:458] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:58:19.981878 10713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:58:19.983258 10713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:58:19.986297 10713 catalog_manager.cc:1347] Generated new cluster ID: 5618073336fc4d47be207d94e53c50fe
I20260812 03:58:19.986404 10713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:58:20.011006 10713 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:58:20.011258 10713 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:58:20.020190 10713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e401c743e7a24ee0a5084fd4a15710c1: Generated new TSK 0
I20260812 03:58:20.020493 10713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:58:20.025082 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:41181
--builtin_ntp_servers=127.31.43.212:45539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:58:20.122941 10726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:20.123211 10726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:20.123255 10726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:20.124627 10726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:20.124706 10726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:20.124751 10726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:58:20.126286 10726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:20.126614 10726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:20.126997 10726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:20.133212 10733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:20.133284 10726 server_base.cc:1034] running on GCE node
W20260812 03:58:20.133302 10731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:20.133212 10735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:20.133598 10726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:20.133857 10726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:20.135074 10726 hybrid_clock.cc:648] HybridClock initialized: now 1786507100135059 us; error 34 us; skew 500 ppm
I20260812 03:58:20.136533 10726 webserver.cc:460] Webserver started at http://127.31.43.193:39071/ using document root <none> and password file <none>
I20260812 03:58:20.136847 10726 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:20.136929 10726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:20.137106 10726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:20.138430 10726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "40c7aaa6d12142c8b3baa38419e3a860"
format_stamp: "Formatted at 2026-08-12 03:58:20 on dist-test-slave-8tdl"
server_key: "39d607c94cd9638acc60f68c95f4455b"
server_key_iv: "8b3144306336e6f2074f3c21d1bb7318"
server_key_version: "encryptionkey@0"
I20260812 03:58:20.138877 10726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "40c7aaa6d12142c8b3baa38419e3a860"
format_stamp: "Formatted at 2026-08-12 03:58:20 on dist-test-slave-8tdl"
server_key: "39d607c94cd9638acc60f68c95f4455b"
server_key_iv: "8b3144306336e6f2074f3c21d1bb7318"
server_key_version: "encryptionkey@0"
I20260812 03:58:20.140487 10726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:58:20.141491 10744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:20.141752 10726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:58:20.141901 10726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "40c7aaa6d12142c8b3baa38419e3a860"
format_stamp: "Formatted at 2026-08-12 03:58:20 on dist-test-slave-8tdl"
server_key: "39d607c94cd9638acc60f68c95f4455b"
server_key_iv: "8b3144306336e6f2074f3c21d1bb7318"
server_key_version: "encryptionkey@0"
I20260812 03:58:20.142002 10726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:20.146191 10726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:20.146572 10726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:20.146771 10726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:20.147050 10726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:20.147424 10726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:20.147496 10726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:20.147544 10726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:20.147572 10726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:20.155032 10726 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:40123
I20260812 03:58:20.155084 10877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:40123 every 8 connection(s)
I20260812 03:58:20.155503 10726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:58:20.159152 10701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:20.156400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:47795 (local address 127.31.43.254:41181)
0812 03:58:20.156488 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:20.156491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:20.156749 (+ 258us) server_negotiation.cc:407] Connection header received
0812 03:58:20.157354 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:20.157355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:20.157361 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:20.157435 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:20.158812 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:20.158813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:20.158813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:58:20.158819 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:20.158839 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:58:20.158845 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:20.158986 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:20.159071 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 03:58:20.159087 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:58:20.159452 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:20.156418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:41181 (local address 127.31.43.193:47795)
0812 03:58:20.156626 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:20.156664 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:20.157205 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:20.157554 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:20.157558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:20.157567 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:20.158573 (+ 1006us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:58:20.158574 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:20.158606 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:58:20.158638 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:58:20.158999 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:58:20.159006 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:58:20.159008 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:58:20.159091 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:58:20.159151 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260812 03:58:20.160302 10879 heartbeater.cc:347] Connected to a master server at 127.31.43.254:41181
I20260812 03:58:20.160527 10879 heartbeater.cc:464] Registering TS with master...
I20260812 03:58:20.160703 10879 heartbeater.cc:511] Master 127.31.43.254:41181 requested a full tablet report, sending...
I20260812 03:58:20.161356 10658 ts_manager.cc:194] Registered new tserver with Master: 40c7aaa6d12142c8b3baa38419e3a860 (127.31.43.193:40123)
I20260812 03:58:20.162204 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 10726
I20260812 03:58:20.162300 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:58:20.162470 31919 external_mini_cluster.cc:1428] Setting key 13fc2de366f349a0e64adca6bfde6f71
I20260812 03:58:20.163717 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:41181
--builtin_ntp_servers=127.31.43.212:45539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:58:20.252060 10882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:20.252300 10882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:20.252326 10882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:20.253530 10882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:20.253572 10882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:20.253657 10882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:58:20.255308 10882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:20.255618 10882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:20.255949 10882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:20.261719 10887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:20.261677 10889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:20.261678 10891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:20.261965 10882 server_base.cc:1034] running on GCE node
I20260812 03:58:20.262244 10882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:20.262486 10882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:20.263653 10882 hybrid_clock.cc:648] HybridClock initialized: now 1786507100263632 us; error 51 us; skew 500 ppm
I20260812 03:58:20.265164 10882 webserver.cc:460] Webserver started at http://127.31.43.194:36249/ using document root <none> and password file <none>
I20260812 03:58:20.265494 10882 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:20.265566 10882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:20.265733 10882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:20.267186 10882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "0f911631d7574db4b9049e20d5c5156e"
format_stamp: "Formatted at 2026-08-12 03:58:20 on dist-test-slave-8tdl"
server_key: "c8acbfe1f8c63fac383907fe4f50ce55"
server_key_iv: "7b0d2ae2c912eaea571ce246bf299812"
server_key_version: "encryptionkey@0"
I20260812 03:58:20.267621 10882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "0f911631d7574db4b9049e20d5c5156e"
format_stamp: "Formatted at 2026-08-12 03:58:20 on dist-test-slave-8tdl"
server_key: "c8acbfe1f8c63fac383907fe4f50ce55"
server_key_iv: "7b0d2ae2c912eaea571ce246bf299812"
server_key_version: "encryptionkey@0"
I20260812 03:58:20.269274 10882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:58:20.270277 10898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:20.270509 10882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:20.270604 10882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "0f911631d7574db4b9049e20d5c5156e"
format_stamp: "Formatted at 2026-08-12 03:58:20 on dist-test-slave-8tdl"
server_key: "c8acbfe1f8c63fac383907fe4f50ce55"
server_key_iv: "7b0d2ae2c912eaea571ce246bf299812"
server_key_version: "encryptionkey@0"
I20260812 03:58:20.270757 10882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:20.274982 10882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:20.275341 10882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:20.275511 10882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:20.275786 10882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:20.276196 10882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:20.276234 10882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:20.276298 10882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:20.276338 10882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:20.283838 10882 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:37407
I20260812 03:58:20.283878 11027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:37407 every 8 connection(s)
I20260812 03:58:20.284292 10882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:58:20.287909 10701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:20.285215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:39715 (local address 127.31.43.254:41181)
0812 03:58:20.285292 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:20.285296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:20.285601 (+ 305us) server_negotiation.cc:407] Connection header received
0812 03:58:20.286197 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:20.286199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:20.286206 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:20.286273 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:20.287649 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:20.287650 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:20.287651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:58:20.287656 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:20.287678 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:58:20.287682 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:20.287754 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:20.287819 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 03:58:20.287834 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:58:20.288134 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:20.285226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:41181 (local address 127.31.43.194:39715)
0812 03:58:20.285461 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:20.285499 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:20.286009 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:20.286359 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:20.286362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:20.286371 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:20.287454 (+ 1083us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:58:20.287455 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:20.287490 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:58:20.287524 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:58:20.287751 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:58:20.287755 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:58:20.287757 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:58:20.287814 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:58:20.287876 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
I20260812 03:58:20.288877 11028 heartbeater.cc:347] Connected to a master server at 127.31.43.254:41181
I20260812 03:58:20.289026 11028 heartbeater.cc:464] Registering TS with master...
I20260812 03:58:20.289316 11028 heartbeater.cc:511] Master 127.31.43.254:41181 requested a full tablet report, sending...
I20260812 03:58:20.289875 10658 ts_manager.cc:194] Registered new tserver with Master: 0f911631d7574db4b9049e20d5c5156e (127.31.43.194:37407)
I20260812 03:58:20.290458 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 10882
I20260812 03:58:20.290563 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:58:20.290818 31919 external_mini_cluster.cc:1428] Setting key e28695cbd2ec158612132dd4657ae47f
I20260812 03:58:20.291795 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:41181
--builtin_ntp_servers=127.31.43.212:45539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:58:20.380766 11033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:20.381011 11033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:20.381037 11033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:20.382432 11033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:20.382493 11033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:20.382524 11033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:58:20.384126 11033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:20.384423 11033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:20.384764 11033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:20.390475 11040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:20.390497 11038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:20.390486 11042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:20.390756 11033 server_base.cc:1034] running on GCE node
I20260812 03:58:20.391179 11033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:20.391461 11033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:20.392623 11033 hybrid_clock.cc:648] HybridClock initialized: now 1786507100392607 us; error 31 us; skew 500 ppm
I20260812 03:58:20.394200 11033 webserver.cc:460] Webserver started at http://127.31.43.195:39457/ using document root <none> and password file <none>
I20260812 03:58:20.394498 11033 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:20.394555 11033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:20.394807 11033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:20.396024 11033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "aba859ca5e974cc699177dc7cf0748ff"
format_stamp: "Formatted at 2026-08-12 03:58:20 on dist-test-slave-8tdl"
server_key: "2c947f5474e15369867da22aa09b15a6"
server_key_iv: "9e6b8dbfa206fe41d1b24eecab0c86d1"
server_key_version: "encryptionkey@0"
I20260812 03:58:20.396436 11033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "aba859ca5e974cc699177dc7cf0748ff"
format_stamp: "Formatted at 2026-08-12 03:58:20 on dist-test-slave-8tdl"
server_key: "2c947f5474e15369867da22aa09b15a6"
server_key_iv: "9e6b8dbfa206fe41d1b24eecab0c86d1"
server_key_version: "encryptionkey@0"
I20260812 03:58:20.398039 11033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:58:20.399004 11049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:20.399220 11033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:58:20.399326 11033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "aba859ca5e974cc699177dc7cf0748ff"
format_stamp: "Formatted at 2026-08-12 03:58:20 on dist-test-slave-8tdl"
server_key: "2c947f5474e15369867da22aa09b15a6"
server_key_iv: "9e6b8dbfa206fe41d1b24eecab0c86d1"
server_key_version: "encryptionkey@0"
I20260812 03:58:20.399410 11033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:20.403224 11033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:20.403548 11033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:20.403687 11033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:20.403928 11033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:20.404294 11033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:20.404327 11033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:20.404379 11033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:20.404408 11033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:20.410936 11033 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:39753
I20260812 03:58:20.410985 11182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:39753 every 8 connection(s)
I20260812 03:58:20.411404 11033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:58:20.414736 10701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:20.412237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:55435 (local address 127.31.43.254:41181)
0812 03:58:20.412286 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:20.412288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:20.412545 (+ 257us) server_negotiation.cc:407] Connection header received
0812 03:58:20.413038 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:20.413039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:20.413049 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:20.413115 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:20.414374 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:20.414375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:20.414375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:58:20.414379 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:20.414394 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:58:20.414398 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:20.414452 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:20.414606 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:58:20.414622 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:58:20.415027 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:20.412196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:41181 (local address 127.31.43.195:55435)
0812 03:58:20.412403 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:20.412447 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:20.412931 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:20.413196 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:20.413200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:20.413209 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:20.414197 (+ 988us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:58:20.414197 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:20.414230 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:58:20.414264 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:58:20.414505 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:58:20.414513 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:58:20.414516 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:58:20.414609 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 03:58:20.414669 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
I20260812 03:58:20.415657 11183 heartbeater.cc:347] Connected to a master server at 127.31.43.254:41181
I20260812 03:58:20.415784 11183 heartbeater.cc:464] Registering TS with master...
I20260812 03:58:20.416010 11183 heartbeater.cc:511] Master 127.31.43.254:41181 requested a full tablet report, sending...
I20260812 03:58:20.416519 10658 ts_manager.cc:194] Registered new tserver with Master: aba859ca5e974cc699177dc7cf0748ff (127.31.43.195:39753)
I20260812 03:58:20.418524 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 11033
I20260812 03:58:20.418663 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:58:20.418900 31919 external_mini_cluster.cc:1428] Setting key 06be557e5ecb7943ac5788008ab13f8c
I20260812 03:58:20.420281 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:58:20.422533 10701 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:58:20.421813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46356 (local address 127.31.43.254:41181)
0812 03:58:20.421873 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:20.421876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:20.422017 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:58:20.422204 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:20.422205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:20.422211 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:20.422290 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:20.422445 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46356: BlockingRecv error: recv got EOF from 127.0.0.1:46356 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:58:20.422653 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 10726
I20260812 03:58:20.432510 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 10882
I20260812 03:58:20.445708 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 11033
I20260812 03:58:20.453820 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 10621
2026-08-12T03:58:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (699 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:58:20 dist-test-slave-8tdl krb5kdc[11196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:58:20 dist-test-slave-8tdl krb5kdc[11196](info): set up 2 sockets
Aug 12 03:58:20 dist-test-slave-8tdl krb5kdc[11196](info): commencing operation
krb5kdc: starting...
W20260812 03:58:21.588470 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:58:21 dist-test-slave-8tdl krb5kdc[11196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:58:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:58:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:58:21.730466 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:46855
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:46855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:58:21.820631 11211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:21.820879 11211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:21.820905 11211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:21.822187 11211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:58:21.822243 11211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:21.822258 11211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:21.822273 11211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:58:21.822285 11211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:58:21.823994 11211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:46855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:46855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:21.824291 11211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:21.824563 11211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:21.830772 11218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:21.830772 11216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:21.830855 11220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:21.831111 11211 server_base.cc:1034] running on GCE node
I20260812 03:58:21.831395 11211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:21.831725 11211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:21.832957 11211 hybrid_clock.cc:648] HybridClock initialized: now 1786507101832931 us; error 50 us; skew 500 ppm
Aug 12 03:58:21 dist-test-slave-8tdl krb5kdc[11196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507101, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:58:21.835829 11211 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:58:21.836416 11211 webserver.cc:460] Webserver started at http://127.31.43.254:35357/ using document root <none> and password file <none>
I20260812 03:58:21.836683 11211 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:21.836756 11211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:21.836913 11211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:21.838085 11211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "23616f183db1461798fad0672366ff38"
format_stamp: "Formatted at 2026-08-12 03:58:21 on dist-test-slave-8tdl"
server_key: "37a6a0c1a472f4a40a97976c2f3964db"
server_key_iv: "9994f7c891cdd177ae764e0cfdf09bca"
server_key_version: "encryptionkey@0"
I20260812 03:58:21.838476 11211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "23616f183db1461798fad0672366ff38"
format_stamp: "Formatted at 2026-08-12 03:58:21 on dist-test-slave-8tdl"
server_key: "37a6a0c1a472f4a40a97976c2f3964db"
server_key_iv: "9994f7c891cdd177ae764e0cfdf09bca"
server_key_version: "encryptionkey@0"
I20260812 03:58:21.839978 11211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:21.840845 11229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:21.841069 11211 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:21.841183 11211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "23616f183db1461798fad0672366ff38"
format_stamp: "Formatted at 2026-08-12 03:58:21 on dist-test-slave-8tdl"
server_key: "37a6a0c1a472f4a40a97976c2f3964db"
server_key_iv: "9994f7c891cdd177ae764e0cfdf09bca"
server_key_version: "encryptionkey@0"
I20260812 03:58:21.841270 11211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:21.854183 11211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:21.856503 11211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:21.856690 11211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:21.861440 11211 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:46855
I20260812 03:58:21.861519 11294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:46855 every 8 connection(s)
I20260812 03:58:21.861927 11211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:58:21.863075 11295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:58:21.865577 11295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38: Bootstrap starting.
I20260812 03:58:21.866173 11295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38: Neither blocks nor log segments found. Creating new log.
I20260812 03:58:21.866432 11295 log.cc:824] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38: Log is configured to *not* fsync() on all Append() calls
I20260812 03:58:21.867274 11295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38: No bootstrap required, opened a new log
I20260812 03:58:21.867395 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 11211
I20260812 03:58:21.867516 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:58:21.867722 31919 external_mini_cluster.cc:1428] Setting key 1d8c8aeb8e58de8e20bdbd4605134ef1
I20260812 03:58:21.869833 11295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23616f183db1461798fad0672366ff38" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 46855 } }
I20260812 03:58:21.870038 11295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:58:21.870083 11295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23616f183db1461798fad0672366ff38, State: Initialized, Role: FOLLOWER
I20260812 03:58:21.870249 11295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23616f183db1461798fad0672366ff38" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 46855 } }
I20260812 03:58:21.870344 11295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:58:21.870409 11295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:58:21.870461 11295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:58:21.871834 11295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23616f183db1461798fad0672366ff38" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 46855 } }
I20260812 03:58:21.871960 11295 leader_election.cc:302] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23616f183db1461798fad0672366ff38; no voters:
I20260812 03:58:21.872133 11295 leader_election.cc:288] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:58:21.872216 11300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:58:21.872385 11300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [term 1 LEADER]: Becoming Leader. State: Replica: 23616f183db1461798fad0672366ff38, State: Running, Role: LEADER
I20260812 03:58:21.872423 11295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:58:21.872478 11300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23616f183db1461798fad0672366ff38" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 46855 } }
I20260812 03:58:21.872829 11301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23616f183db1461798fad0672366ff38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23616f183db1461798fad0672366ff38" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 46855 } } }
I20260812 03:58:21.872933 11301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [sys.catalog]: This master's current role is: LEADER
I20260812 03:58:21.873416 11307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:58:21.874481 11307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:58:21.873349 11302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23616f183db1461798fad0672366ff38. Latest consensus state: current_term: 1 leader_uuid: "23616f183db1461798fad0672366ff38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23616f183db1461798fad0672366ff38" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 46855 } } }
I20260812 03:58:21.875306 11302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38 [sys.catalog]: This master's current role is: LEADER
I20260812 03:58:21.876390 11307 catalog_manager.cc:1347] Generated new cluster ID: 202cbec4208c4d9fbb4e835dccc3775b
I20260812 03:58:21.876447 11307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:58:21 dist-test-slave-8tdl krb5kdc[11196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:58:21.881019 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:21.868403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35390 (local address 127.31.43.254:46855)
0812 03:58:21.868664 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:21.868674 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:21.868710 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:58:21.869783 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:21.869793 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:21.869865 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:21.870040 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:21.870953 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:21.872445 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:21.873569 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:21.874225 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:21.874347 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:21.877348 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:21.877352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:21.877352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:58:21.877366 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:21.879293 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:21.879718 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:21.879722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:21.879722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:21.879846 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:21.880080 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:21.880081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:21.880082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:21.880346 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:21.880538 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:21.880636 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 03:58:21.880788 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":140,"threads_started":1}
I20260812 03:58:21.923470 11307 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:58:21.924259 11307 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:58:21.935077 11307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38: Generated new TSK 0
I20260812 03:58:21.935338 11307 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 03:58:21.975721 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:46855
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:58:22.070117 11321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:22.070382 11321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:22.070411 11321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:22.071820 11321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:22.071955 11321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:22.072230 11321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:58:22.074633 11321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:46855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:22.075163 11321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:22.075663 11321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:22.081954 11328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:22.081974 11321 server_base.cc:1034] running on GCE node
W20260812 03:58:22.081945 11329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:22.082083 11331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:22.082574 11321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:22.082890 11321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:22.084064 11321 hybrid_clock.cc:648] HybridClock initialized: now 1786507102084050 us; error 30 us; skew 500 ppm
Aug 12 03:58:22 dist-test-slave-8tdl krb5kdc[11196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:58:22.087179 11321 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 03:58:22.087708 11321 webserver.cc:460] Webserver started at http://127.31.43.193:39337/ using document root <none> and password file <none>
I20260812 03:58:22.087960 11321 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:22.088037 11321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:22.088200 11321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:22.089375 11321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "4d3d9b41c99f4df48f41adef8f300841"
format_stamp: "Formatted at 2026-08-12 03:58:22 on dist-test-slave-8tdl"
server_key: "a57952128049c300cc1374c1f495f39e"
server_key_iv: "d736dbf65948dd75219b860bfa596373"
server_key_version: "encryptionkey@0"
I20260812 03:58:22.089782 11321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "4d3d9b41c99f4df48f41adef8f300841"
format_stamp: "Formatted at 2026-08-12 03:58:22 on dist-test-slave-8tdl"
server_key: "a57952128049c300cc1374c1f495f39e"
server_key_iv: "d736dbf65948dd75219b860bfa596373"
server_key_version: "encryptionkey@0"
I20260812 03:58:22.091565 11321 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:58:22.092687 11338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:22.092916 11321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:22.093067 11321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "4d3d9b41c99f4df48f41adef8f300841"
format_stamp: "Formatted at 2026-08-12 03:58:22 on dist-test-slave-8tdl"
server_key: "a57952128049c300cc1374c1f495f39e"
server_key_iv: "d736dbf65948dd75219b860bfa596373"
server_key_version: "encryptionkey@0"
I20260812 03:58:22.093168 11321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:22.100695 11321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:22.103075 11321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:22.103286 11321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:22.103570 11321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:22.103924 11321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:22.103981 11321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:22.104039 11321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:22.104069 11321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:22.110663 11321 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:43345
I20260812 03:58:22.110736 11473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:43345 every 8 connection(s)
I20260812 03:58:22.111128 11321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 03:58:22.113106 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 11321
I20260812 03:58:22.113236 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:58:22.113446 31919 external_mini_cluster.cc:1428] Setting key 8f537838aa63e92ae6395eebdebfd9b4
Aug 12 03:58:22 dist-test-slave-8tdl krb5kdc[11196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
I20260812 03:58:22.123751 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.112028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:54795 (local address 127.31.43.254:46855)
0812 03:58:22.112115 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:22.112117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:22.112365 (+ 248us) server_negotiation.cc:407] Connection header received
0812 03:58:22.113049 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:22.113050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:22.113068 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:22.113136 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:22.114668 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.115281 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.116793 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.117057 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.117148 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.120179 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:22.120181 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:22.120181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:58:22.120190 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:22.122018 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:22.122565 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:22.122567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:22.122568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:22.122638 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:22.122963 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:22.122964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:22.122965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:22.123108 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:22.123192 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:22.123526 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 03:58:22.123647 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:58:22.123965 11476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.111983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:46855 (local address 127.31.43.193:54795)
0812 03:58:22.112231 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:22.112274 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:22.112909 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:22.113254 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:22.113258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:22.114131 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:22.114469 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.114478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.115394 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.115395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.116689 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.116693 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.117180 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.117181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.117320 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.118069 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:22.118078 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:22.120032 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:58:22.122159 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:58:22.122172 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:58:22.122175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:58:22.122474 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:58:22.122748 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:58:22.122749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:58:22.122749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:58:22.122886 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:58:22.123182 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:58:22.123191 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:58:22.123371 (+ 180us) client_negotiation.cc:769] Sending connection context
0812 03:58:22.123502 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 03:58:22.123695 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":145,"threads_started":1}
I20260812 03:58:22.124655 11474 heartbeater.cc:347] Connected to a master server at 127.31.43.254:46855
I20260812 03:58:22.124773 11474 heartbeater.cc:464] Registering TS with master...
I20260812 03:58:22.124991 11474 heartbeater.cc:511] Master 127.31.43.254:46855 requested a full tablet report, sending...
I20260812 03:58:22.125730 11250 ts_manager.cc:194] Registered new tserver with Master: 4d3d9b41c99f4df48f41adef8f300841 (127.31.43.193:43345)
I20260812 03:58:22.126518 11250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:54795
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 03:58:22.146970 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:46855
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:58:22.236421 11480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:22.236682 11480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:22.236728 11480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:22.238065 11480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:22.238132 11480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:22.238179 11480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:58:22.239913 11480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:46855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:22.240201 11480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:22.240465 11480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:22.246152 11486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:22.246152 11489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:22.246193 11480 server_base.cc:1034] running on GCE node
W20260812 03:58:22.246148 11487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:22.246662 11480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:22.247037 11480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:22.248208 11480 hybrid_clock.cc:648] HybridClock initialized: now 1786507102248203 us; error 45 us; skew 500 ppm
Aug 12 03:58:22 dist-test-slave-8tdl krb5kdc[11196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:58:22.251291 11480 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 03:58:22.251852 11480 webserver.cc:460] Webserver started at http://127.31.43.194:46351/ using document root <none> and password file <none>
I20260812 03:58:22.252133 11480 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:22.252213 11480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:22.252375 11480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:22.253698 11480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "68f50bc57aed4d42b1771a4289515497"
format_stamp: "Formatted at 2026-08-12 03:58:22 on dist-test-slave-8tdl"
server_key: "1abdfa00fcaee84115338c6206d84cc6"
server_key_iv: "ff8c7177d7995c9449501917e6d477ec"
server_key_version: "encryptionkey@0"
I20260812 03:58:22.254110 11480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "68f50bc57aed4d42b1771a4289515497"
format_stamp: "Formatted at 2026-08-12 03:58:22 on dist-test-slave-8tdl"
server_key: "1abdfa00fcaee84115338c6206d84cc6"
server_key_iv: "ff8c7177d7995c9449501917e6d477ec"
server_key_version: "encryptionkey@0"
I20260812 03:58:22.255692 11480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 03:58:22.256598 11497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:22.256907 11480 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:58:22.257077 11480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "68f50bc57aed4d42b1771a4289515497"
format_stamp: "Formatted at 2026-08-12 03:58:22 on dist-test-slave-8tdl"
server_key: "1abdfa00fcaee84115338c6206d84cc6"
server_key_iv: "ff8c7177d7995c9449501917e6d477ec"
server_key_version: "encryptionkey@0"
I20260812 03:58:22.257161 11480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:22.272986 11480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:22.275863 11480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:22.276026 11480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:22.276367 11480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:22.276871 11480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:22.276919 11480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:22.276944 11480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:22.276958 11480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:22.283819 11480 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:37721
I20260812 03:58:22.283890 11629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:37721 every 8 connection(s)
I20260812 03:58:22.284276 11480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:58:22 dist-test-slave-8tdl krb5kdc[11196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:58:22.294111 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 11480
I20260812 03:58:22.294260 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:58:22.294490 31919 external_mini_cluster.cc:1428] Setting key 3097d02ad684c26b3f19a6482cf266ec
I20260812 03:58:22.295295 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.285275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:59797 (local address 127.31.43.254:46855)
0812 03:58:22.285378 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:22.285380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:22.285604 (+ 224us) server_negotiation.cc:407] Connection header received
0812 03:58:22.286178 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:22.286180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:22.286192 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:22.286265 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:22.287417 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.288057 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.289156 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.289381 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.289459 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.292144 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:22.292146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:22.292146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:58:22.292156 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:22.293601 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:22.294081 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:22.294084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:22.294090 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:22.294164 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:22.294492 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:22.294493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:22.294494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:22.294717 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:22.294786 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:22.295119 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 03:58:22.295222 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 03:58:22.295737 11633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.285223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:46855 (local address 127.31.43.194:59797)
0812 03:58:22.285437 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:22.285477 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:22.286047 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:22.286346 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:22.286350 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:22.287075 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:22.287290 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.287295 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.288166 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.288167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.289035 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.289038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.289479 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.289480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.289622 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.290264 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:22.290273 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:22.291991 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:58:22.293698 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:58:22.293707 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:58:22.293709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:58:22.293964 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:58:22.294242 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:58:22.294243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:58:22.294244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:58:22.294416 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:58:22.294788 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:58:22.294798 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:58:22.294961 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 03:58:22.295102 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 03:58:22.295376 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":125,"threads_started":1}
I20260812 03:58:22.296401 11631 heartbeater.cc:347] Connected to a master server at 127.31.43.254:46855
I20260812 03:58:22.296528 11631 heartbeater.cc:464] Registering TS with master...
I20260812 03:58:22.296746 11631 heartbeater.cc:511] Master 127.31.43.254:46855 requested a full tablet report, sending...
I20260812 03:58:22.297289 11250 ts_manager.cc:194] Registered new tserver with Master: 68f50bc57aed4d42b1771a4289515497 (127.31.43.194:37721)
I20260812 03:58:22.297873 11250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:59797
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 03:58:22.327445 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:46855
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:58:22.418314 11636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:58:22.418574 11636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:58:22.418603 11636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:58:22.419930 11636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:58:22.419992 11636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:58:22.420023 11636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:58:22.421677 11636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:46855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:58:22.421983 11636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:58:22.422259 11636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:58:22.428154 11643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:22.428324 11646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:58:22.428601 11644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:58:22.428720 11636 server_base.cc:1034] running on GCE node
I20260812 03:58:22.428987 11636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:58:22.429236 11636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:58:22.430437 11636 hybrid_clock.cc:648] HybridClock initialized: now 1786507102430405 us; error 38 us; skew 500 ppm
Aug 12 03:58:22 dist-test-slave-8tdl krb5kdc[11196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:58:22.433470 11636 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 03:58:22.434032 11636 webserver.cc:460] Webserver started at http://127.31.43.195:45677/ using document root <none> and password file <none>
I20260812 03:58:22.434238 11636 fs_manager.cc:359] Metadata directory not provided
I20260812 03:58:22.434283 11636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:58:22.434381 11636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:58:22.435626 11636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "d5a5f75b78894fd6b325b7442b9f8bcf"
format_stamp: "Formatted at 2026-08-12 03:58:22 on dist-test-slave-8tdl"
server_key: "9482f3b63a0d4971d3ff5eb8ebee7475"
server_key_iv: "c565f783514dc7ee0c90027c71bcf1ad"
server_key_version: "encryptionkey@0"
I20260812 03:58:22.436012 11636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "d5a5f75b78894fd6b325b7442b9f8bcf"
format_stamp: "Formatted at 2026-08-12 03:58:22 on dist-test-slave-8tdl"
server_key: "9482f3b63a0d4971d3ff5eb8ebee7475"
server_key_iv: "c565f783514dc7ee0c90027c71bcf1ad"
server_key_version: "encryptionkey@0"
I20260812 03:58:22.437602 11636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:58:22.438544 11653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:22.438817 11636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:58:22.438954 11636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "d5a5f75b78894fd6b325b7442b9f8bcf"
format_stamp: "Formatted at 2026-08-12 03:58:22 on dist-test-slave-8tdl"
server_key: "9482f3b63a0d4971d3ff5eb8ebee7475"
server_key_iv: "c565f783514dc7ee0c90027c71bcf1ad"
server_key_version: "encryptionkey@0"
I20260812 03:58:22.439086 11636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:58:22.460245 11636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:58:22.463084 11636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:58:22.463279 11636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:58:22.463570 11636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:58:22.463945 11636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:58:22.464001 11636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:22.464036 11636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:58:22.464054 11636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:58:22.470897 11636 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:42377
I20260812 03:58:22.470996 11781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:42377 every 8 connection(s)
I20260812 03:58:22.471371 11636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:58:22.474576 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 11636
I20260812 03:58:22.474730 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:58:22.474937 31919 external_mini_cluster.cc:1428] Setting key bea8d99c1027635bf9d57492c1c45e5f
Aug 12 03:58:22 dist-test-slave-8tdl krb5kdc[11196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:58:22.483425 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.472225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:53481 (local address 127.31.43.254:46855)
0812 03:58:22.472273 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:22.472276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:22.472556 (+ 280us) server_negotiation.cc:407] Connection header received
0812 03:58:22.473148 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:22.473149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:22.473165 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:22.473250 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:22.474408 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.475132 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.476483 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.476706 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.476785 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.479957 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:22.479959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:22.479959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:58:22.479969 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:22.481692 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:22.482237 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:22.482238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:22.482239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:22.482305 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:22.482642 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:22.482643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:22.482644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:22.482857 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:22.482920 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:22.483261 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 03:58:22.483348 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:58:22.483625 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.472189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:46855 (local address 127.31.43.195:53481)
0812 03:58:22.472365 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:22.472412 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:22.473030 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:22.473359 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:22.473363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:22.474051 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:58:22.474273 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.474279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.475245 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.475246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.476383 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.476386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.476808 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.476809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.476954 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.477671 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:58:22.477683 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:58:22.479688 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:58:22.481809 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:58:22.481818 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:58:22.481821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:58:22.482132 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:58:22.482412 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:58:22.482413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:58:22.482413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:58:22.482558 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:58:22.482934 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:58:22.482941 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:58:22.483091 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 03:58:22.483200 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 03:58:22.483362 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260812 03:58:22.484223 11782 heartbeater.cc:347] Connected to a master server at 127.31.43.254:46855
I20260812 03:58:22.484333 11782 heartbeater.cc:464] Registering TS with master...
I20260812 03:58:22.484508 11782 heartbeater.cc:511] Master 127.31.43.254:46855 requested a full tablet report, sending...
I20260812 03:58:22.485023 11250 ts_manager.cc:194] Registered new tserver with Master: d5a5f75b78894fd6b325b7442b9f8bcf (127.31.43.195:42377)
I20260812 03:58:22.485396 11250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:53481
I20260812 03:58:22.487470 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:58:22.495939 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.488786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35404 (local address 127.31.43.254:46855)
0812 03:58:22.488830 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:22.488832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:22.489059 (+ 227us) server_negotiation.cc:407] Connection header received
0812 03:58:22.489151 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:22.489152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:22.489163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:22.489240 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:58:22.489895 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.490418 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.491258 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.491467 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.491547 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.493075 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:58:22.493077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:58:22.493078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:58:22.493087 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:58:22.494404 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:22.494934 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:22.494936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:22.494936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:22.495005 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:58:22.495246 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:58:22.495247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:58:22.495247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:58:22.495442 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:58:22.495560 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:22.495705 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:58:22.495818 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:58:22.498234 11250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:58:22.500135 11250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:58:22.508200 11799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.503735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.31.43.193:43345)
0812 03:58:22.504046 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:22.504051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:22.504199 (+ 148us) server_negotiation.cc:407] Connection header received
0812 03:58:22.504252 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:22.504255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:22.504272 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:22.504394 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:58:22.504637 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.505517 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.506964 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.507720 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.507811 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.507879 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:22.508054 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 03:58:22.508100 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":121,"threads_started":1}
I20260812 03:58:22.508188 11800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.503602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:43345 (local address 127.0.0.1:46564)
0812 03:58:22.504031 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:22.504045 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:22.504114 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:22.504414 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:22.504418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:22.504420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:58:22.504526 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.504529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.505625 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.505625 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.506854 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.506858 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.507812 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.507813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.507902 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.507918 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 03:58:22.508031 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 03:58:22.508105 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":103,"threads_started":1}
I20260812 03:58:22.508555 11801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.503990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55156 (local address 127.31.43.195:42377)
0812 03:58:22.504557 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:22.504561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:22.504569 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:58:22.504622 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:22.504624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:22.504639 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:22.504720 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:58:22.504853 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.506034 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.507300 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.508008 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.508083 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.508134 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:22.508385 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 03:58:22.508438 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":96,"threads_started":1}
I20260812 03:58:22.508771 11798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.503801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:42377 (local address 127.0.0.1:55156)
0812 03:58:22.504406 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:22.504416 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:22.504461 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:22.504737 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:22.504738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:22.504740 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:58:22.504785 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.504787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.506176 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.506177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.507207 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.507209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.508172 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.508174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.508238 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.508243 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:58:22.508647 (+ 404us) client_negotiation.cc:240] Negotiation successful
0812 03:58:22.508703 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":56,"threads_started":1}
I20260812 03:58:22.509136 11399 tablet_service.cc:1467] Processing CreateTablet for tablet 1663e4028be946ca90bc61dcc18a19b4 (DEFAULT_TABLE table=test-table [id=f760e01584fe4ef58c9b1547e81e48be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:58:22.509491 11399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1663e4028be946ca90bc61dcc18a19b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:58:22.509469 11702 tablet_service.cc:1467] Processing CreateTablet for tablet 1663e4028be946ca90bc61dcc18a19b4 (DEFAULT_TABLE table=test-table [id=f760e01584fe4ef58c9b1547e81e48be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:58:22.509797 11702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1663e4028be946ca90bc61dcc18a19b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:58:22.513015 11803 tablet_bootstrap.cc:492] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841: Bootstrap starting.
I20260812 03:58:22.512923 11802 tablet_bootstrap.cc:492] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf: Bootstrap starting.
I20260812 03:58:22.514017 11802 tablet_bootstrap.cc:654] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf: Neither blocks nor log segments found. Creating new log.
I20260812 03:58:22.514038 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.503369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:37721 (local address 127.0.0.1:43426)
0812 03:58:22.503523 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:22.503575 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:22.503660 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:22.504634 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:22.504635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:22.504637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:58:22.504704 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.504706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.506409 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.506409 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.507437 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.507440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.513669 (+ 6229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.513671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.513776 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.513782 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:58:22.513858 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:58:22.513932 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":59,"threads_started":1}
I20260812 03:58:22.513990 11803 tablet_bootstrap.cc:654] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841: Neither blocks nor log segments found. Creating new log.
I20260812 03:58:22.514263 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.503568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43426 (local address 127.31.43.194:37721)
0812 03:58:22.504269 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:22.504274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:22.504285 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:58:22.504340 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:22.504342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:22.504539 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:22.504622 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:58:22.504810 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.506214 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.507535 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.513541 (+ 6006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.513647 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.513729 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:22.513891 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 03:58:22.513968 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":128,"threads_started":1}
I20260812 03:58:22.514829 11803 log.cc:824] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841: Log is configured to *not* fsync() on all Append() calls
I20260812 03:58:22.514832 11802 log.cc:824] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf: Log is configured to *not* fsync() on all Append() calls
I20260812 03:58:22.515278 11552 tablet_service.cc:1467] Processing CreateTablet for tablet 1663e4028be946ca90bc61dcc18a19b4 (DEFAULT_TABLE table=test-table [id=f760e01584fe4ef58c9b1547e81e48be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:58:22.515708 11552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1663e4028be946ca90bc61dcc18a19b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:58:22.515813 11803 tablet_bootstrap.cc:492] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841: No bootstrap required, opened a new log
I20260812 03:58:22.515851 11802 tablet_bootstrap.cc:492] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf: No bootstrap required, opened a new log
I20260812 03:58:22.515900 11803 ts_tablet_manager.cc:1397] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:58:22.515918 11802 ts_tablet_manager.cc:1397] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:58:22.518062 11802 raft_consensus.cc:348] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f50bc57aed4d42b1771a4289515497" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37721 } } peers { permanent_uuid: "4d3d9b41c99f4df48f41adef8f300841" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 43345 } } peers { permanent_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42377 } }
I20260812 03:58:22.518062 11803 raft_consensus.cc:348] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f50bc57aed4d42b1771a4289515497" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37721 } } peers { permanent_uuid: "4d3d9b41c99f4df48f41adef8f300841" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 43345 } } peers { permanent_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42377 } }
I20260812 03:58:22.518268 11802 raft_consensus.cc:374] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:58:22.518268 11803 raft_consensus.cc:374] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:58:22.518307 11803 raft_consensus.cc:729] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d3d9b41c99f4df48f41adef8f300841, State: Initialized, Role: FOLLOWER
I20260812 03:58:22.518307 11802 raft_consensus.cc:729] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a5f75b78894fd6b325b7442b9f8bcf, State: Initialized, Role: FOLLOWER
I20260812 03:58:22.518440 11803 consensus_queue.cc:260] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f50bc57aed4d42b1771a4289515497" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37721 } } peers { permanent_uuid: "4d3d9b41c99f4df48f41adef8f300841" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 43345 } } peers { permanent_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42377 } }
I20260812 03:58:22.518457 11802 consensus_queue.cc:260] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f50bc57aed4d42b1771a4289515497" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37721 } } peers { permanent_uuid: "4d3d9b41c99f4df48f41adef8f300841" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 43345 } } peers { permanent_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42377 } }
I20260812 03:58:22.518836 11803 ts_tablet_manager.cc:1428] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:58:22.518846 11474 heartbeater.cc:503] Master 127.31.43.254:46855 was elected leader, sending a full tablet report...
I20260812 03:58:22.518977 11802 ts_tablet_manager.cc:1428] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 03:58:22.519107 11782 heartbeater.cc:503] Master 127.31.43.254:46855 was elected leader, sending a full tablet report...
I20260812 03:58:22.519125 11806 tablet_bootstrap.cc:492] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497: Bootstrap starting.
I20260812 03:58:22.520061 11806 tablet_bootstrap.cc:654] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497: Neither blocks nor log segments found. Creating new log.
I20260812 03:58:22.520704 11806 log.cc:824] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497: Log is configured to *not* fsync() on all Append() calls
I20260812 03:58:22.521574 11806 tablet_bootstrap.cc:492] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497: No bootstrap required, opened a new log
I20260812 03:58:22.521667 11806 ts_tablet_manager.cc:1397] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:58:22.523258 11806 raft_consensus.cc:348] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f50bc57aed4d42b1771a4289515497" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37721 } } peers { permanent_uuid: "4d3d9b41c99f4df48f41adef8f300841" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 43345 } } peers { permanent_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42377 } }
I20260812 03:58:22.523444 11806 raft_consensus.cc:374] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:58:22.523494 11806 raft_consensus.cc:729] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f50bc57aed4d42b1771a4289515497, State: Initialized, Role: FOLLOWER
I20260812 03:58:22.523658 11806 consensus_queue.cc:260] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f50bc57aed4d42b1771a4289515497" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37721 } } peers { permanent_uuid: "4d3d9b41c99f4df48f41adef8f300841" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 43345 } } peers { permanent_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42377 } }
I20260812 03:58:22.523967 11806 ts_tablet_manager.cc:1428] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:58:22.524156 11631 heartbeater.cc:503] Master 127.31.43.254:46855 was elected leader, sending a full tablet report...
W20260812 03:58:22.534978 11632 tablet.cc:2369] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:58:22.612103 11475 tablet.cc:2369] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:58:22.673883 11808 raft_consensus.cc:482] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:58:22.674095 11808 raft_consensus.cc:504] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f50bc57aed4d42b1771a4289515497" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37721 } } peers { permanent_uuid: "4d3d9b41c99f4df48f41adef8f300841" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 43345 } } peers { permanent_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42377 } }
I20260812 03:58:22.674561 11808 leader_election.cc:288] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f50bc57aed4d42b1771a4289515497 (127.31.43.194:37721), 4d3d9b41c99f4df48f41adef8f300841 (127.31.43.193:43345)
I20260812 03:58:22.679091 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.674846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:47467 (local address 127.31.43.194:37721)
0812 03:58:22.674936 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:22.674938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:22.675167 (+ 229us) server_negotiation.cc:407] Connection header received
0812 03:58:22.675245 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:22.675246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:22.675260 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:22.675321 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:58:22.675543 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.676363 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.677991 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.678604 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.678726 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.678738 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:22.678966 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 03:58:22.679016 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:58:22.679091 11799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.675343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:56683 (local address 127.31.43.193:43345)
0812 03:58:22.675393 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:22.675394 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:22.675402 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:58:22.675434 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:22.675435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:22.675442 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:22.675485 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:58:22.675668 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.676362 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.677992 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.678615 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.678732 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.678741 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:22.678966 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 03:58:22.679016 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:58:22.679069 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.674785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:37721 (local address 127.31.43.195:47467)
0812 03:58:22.675111 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:22.675124 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:22.675184 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:22.675347 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:22.675348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:22.675351 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:58:22.675435 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.675437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.676467 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.676468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.677821 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.677826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.678742 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.678743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.678835 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.678837 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:58:22.678902 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:58:22.678963 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":86}
I20260812 03:58:22.679075 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.674936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:43345 (local address 127.31.43.195:56683)
0812 03:58:22.675197 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:22.675210 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:58:22.675291 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:58:22.675516 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:58:22.675517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:58:22.675519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:58:22.675589 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.675590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.676467 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.676468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.677821 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:58:22.677826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.678735 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.678736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:58:22.678831 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.678837 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:58:22.678902 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:58:22.678963 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":58,"thread_start_us":126,"threads_started":1}
I20260812 03:58:22.679916 11577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1663e4028be946ca90bc61dcc18a19b4" candidate_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f50bc57aed4d42b1771a4289515497" is_pre_election: true
I20260812 03:58:22.679919 11423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1663e4028be946ca90bc61dcc18a19b4" candidate_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d3d9b41c99f4df48f41adef8f300841" is_pre_election: true
I20260812 03:58:22.680104 11577 raft_consensus.cc:2454] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a5f75b78894fd6b325b7442b9f8bcf in term 0.
I20260812 03:58:22.680104 11423 raft_consensus.cc:2454] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a5f75b78894fd6b325b7442b9f8bcf in term 0.
I20260812 03:58:22.680330 11655 leader_election.cc:302] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f50bc57aed4d42b1771a4289515497, d5a5f75b78894fd6b325b7442b9f8bcf; no voters:
I20260812 03:58:22.680471 11808 raft_consensus.cc:2781] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:58:22.680529 11808 raft_consensus.cc:482] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:58:22.680554 11808 raft_consensus.cc:3037] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:58:22.681322 11808 raft_consensus.cc:504] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f50bc57aed4d42b1771a4289515497" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37721 } } peers { permanent_uuid: "4d3d9b41c99f4df48f41adef8f300841" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 43345 } } peers { permanent_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42377 } }
I20260812 03:58:22.681478 11808 leader_election.cc:288] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [CANDIDATE]: Term 1 election: Requested vote from peers 68f50bc57aed4d42b1771a4289515497 (127.31.43.194:37721), 4d3d9b41c99f4df48f41adef8f300841 (127.31.43.193:43345)
I20260812 03:58:22.681682 11577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1663e4028be946ca90bc61dcc18a19b4" candidate_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f50bc57aed4d42b1771a4289515497"
I20260812 03:58:22.681687 11423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1663e4028be946ca90bc61dcc18a19b4" candidate_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d3d9b41c99f4df48f41adef8f300841"
I20260812 03:58:22.681780 11577 raft_consensus.cc:3037] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:58:22.681780 11423 raft_consensus.cc:3037] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:58:22.682798 11577 raft_consensus.cc:2454] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a5f75b78894fd6b325b7442b9f8bcf in term 1.
I20260812 03:58:22.682798 11423 raft_consensus.cc:2454] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a5f75b78894fd6b325b7442b9f8bcf in term 1.
I20260812 03:58:22.683074 11655 leader_election.cc:302] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f50bc57aed4d42b1771a4289515497, d5a5f75b78894fd6b325b7442b9f8bcf; no voters:
I20260812 03:58:22.683215 11808 raft_consensus.cc:2781] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:58:22.683382 11808 raft_consensus.cc:686] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 1 LEADER]: Becoming Leader. State: Replica: d5a5f75b78894fd6b325b7442b9f8bcf, State: Running, Role: LEADER
I20260812 03:58:22.683485 11808 consensus_queue.cc:237] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f50bc57aed4d42b1771a4289515497" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37721 } } peers { permanent_uuid: "4d3d9b41c99f4df48f41adef8f300841" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 43345 } } peers { permanent_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42377 } }
I20260812 03:58:22.684492 11248 catalog_manager.cc:5495] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf reported cstate change: term changed from 0 to 1, leader changed from <none> to d5a5f75b78894fd6b325b7442b9f8bcf (127.31.43.195). New cstate: current_term: 1 leader_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f50bc57aed4d42b1771a4289515497" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d3d9b41c99f4df48f41adef8f300841" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 43345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a5f75b78894fd6b325b7442b9f8bcf" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 42377 } health_report { overall_health: HEALTHY } } }
I20260812 03:58:22.692677 11801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.689975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.31.43.195:42377)
0812 03:58:22.690113 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:58:22.690115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:58:22.690123 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:58:22.690166 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:58:22.690167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:58:22.690188 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:58:22.690270 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:58:22.690432 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.690978 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.691810 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:58:22.692023 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:58:22.692114 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:58:22.692312 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:58:22.692415 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:58:22.692506 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:58:22.692529 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 03:58:22.692555 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:58:22.695183 11423 raft_consensus.cc:1261] T 1663e4028be946ca90bc61dcc18a19b4 P 4d3d9b41c99f4df48f41adef8f300841 [term 1 FOLLOWER]: Refusing update from remote peer d5a5f75b78894fd6b325b7442b9f8bcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:58:22.695179 11577 raft_consensus.cc:1261] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497 [term 1 FOLLOWER]: Refusing update from remote peer d5a5f75b78894fd6b325b7442b9f8bcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:58:22.695618 11814 consensus_queue.cc:1035] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d3d9b41c99f4df48f41adef8f300841" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 43345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:58:22.695796 11808 consensus_queue.cc:1035] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f50bc57aed4d42b1771a4289515497" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:58:22.702343 11250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:58:22.702491 11250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:58:22.703542 11250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23616f183db1461798fad0672366ff38: Sending DeleteTablet for 3 replicas of tablet 1663e4028be946ca90bc61dcc18a19b4
I20260812 03:58:22.703953 11552 tablet_service.cc:1514] Processing DeleteTablet for tablet 1663e4028be946ca90bc61dcc18a19b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:58:22 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:43426
I20260812 03:58:22.704015 11399 tablet_service.cc:1514] Processing DeleteTablet for tablet 1663e4028be946ca90bc61dcc18a19b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:58:22 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:46564
I20260812 03:58:22.704015 11702 tablet_service.cc:1514] Processing DeleteTablet for tablet 1663e4028be946ca90bc61dcc18a19b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:58:22 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:55156
I20260812 03:58:22.704252 11829 tablet_replica.cc:330] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497: stopping tablet replica
I20260812 03:58:22.704373 11829 raft_consensus.cc:2229] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:58:22.704514 11829 raft_consensus.cc:2258] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:58:22.704627 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 11321
I20260812 03:58:22.707517 11829 ts_tablet_manager.cc:1905] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:58:22.710666 11829 ts_tablet_manager.cc:1918] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:58:22.710822 11829 log.cc:1196] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/1663e4028be946ca90bc61dcc18a19b4
I20260812 03:58:22.711216 11829 ts_tablet_manager.cc:1939] T 1663e4028be946ca90bc61dcc18a19b4 P 68f50bc57aed4d42b1771a4289515497: Deleting consensus metadata
I20260812 03:58:22.711854 11232 catalog_manager.cc:4841] TS 68f50bc57aed4d42b1771a4289515497 (127.31.43.194:37721): tablet 1663e4028be946ca90bc61dcc18a19b4 (table test-table [id=f760e01584fe4ef58c9b1547e81e48be]) successfully deleted
I20260812 03:58:22.712528 11832 tablet_replica.cc:330] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf: stopping tablet replica
I20260812 03:58:22.712718 11832 raft_consensus.cc:2229] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:58:22.712913 11832 raft_consensus.cc:2258] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:58:22.713828 11832 ts_tablet_manager.cc:1905] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:58:22.715879 11832 ts_tablet_manager.cc:1918] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:58:22.716006 11832 log.cc:1196] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/1663e4028be946ca90bc61dcc18a19b4
I20260812 03:58:22.716396 11832 ts_tablet_manager.cc:1939] T 1663e4028be946ca90bc61dcc18a19b4 P d5a5f75b78894fd6b325b7442b9f8bcf: Deleting consensus metadata
I20260812 03:58:22.716953 11232 catalog_manager.cc:4841] TS d5a5f75b78894fd6b325b7442b9f8bcf (127.31.43.195:42377): tablet 1663e4028be946ca90bc61dcc18a19b4 (table test-table [id=f760e01584fe4ef58c9b1547e81e48be]) successfully deleted
W20260812 03:58:22.720016 11232 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:43345 (error 108)
I20260812 03:58:22.720213 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 11480
I20260812 03:58:22.730036 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:58:22.729830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:43345 (local address 127.0.0.1:46580)
0812 03:58:22.729909 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 03:58:22.729965 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 03:58:22.730206 11232 catalog_manager.cc:4584] TS 4d3d9b41c99f4df48f41adef8f300841 (127.31.43.193:43345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1663e4028be946ca90bc61dcc18a19b4: Network error: Client connection negotiation failed: client connection to 127.31.43.193:43345: connect: Connection refused (error 111)
I20260812 03:58:22.735958 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 11636
I20260812 03:58:22.746762 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 11211
2026-08-12T03:58:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2294 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:58:22 dist-test-slave-8tdl krb5kdc[11838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:58:22 dist-test-slave-8tdl krb5kdc[11838](info): set up 2 sockets
Aug 12 03:58:22 dist-test-slave-8tdl krb5kdc[11838](info): commencing operation
krb5kdc: starting...
W20260812 03:58:23.925150 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:58:24 dist-test-slave-8tdl krb5kdc[11838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:58:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:58:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.43.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.43.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/rangeradmin_127.31.43.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/rangeradmin_127.31.43.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.43.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.43.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/rangerlookup_127.31.43.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/rangerlookup_127.31.43.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.43.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.43.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/HTTP_127.31.43.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/HTTP_127.31.43.212@KRBTEST.COM.keytab.
I20260812 03:58:24.130095 31919 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskaEzvCg/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/postgres -l logfile start
2026-08-12 03:58:27.173 UTC [11872] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 03:58:27.173 UTC [11872] LOG: listening on IPv4 address "127.31.43.212", port 46315
2026-08-12 03:58:27.178 UTC [11872] LOG: listening on Unix socket "/tmp/.s.PGSQL.46315"
2026-08-12 03:58:27.221 UTC [11876] LOG: database system was shut down at 2026-08-12 03:58:25 UTC
2026-08-12 03:58:27.226 UTC [11872] LOG: database system is ready to accept connections
I20260812 03:58:28.255272 31919 mini_postgres.cc:96] Postgres bound to 46315
2026-08-12 03:58:28.265 UTC [11885] FATAL: database "slave" does not exist
127.31.43.212:46315 - accepting connections
I20260812 03:58:28.266582 31919 mini_ranger.cc:161] Starting Ranger...
I20260812 03:58:28.287461 31919 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 03:58:28.957993 31919 mini_ranger.cc:87] Created ranger Postgres database
I20260812 03:58:28.958096 31919 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-admin
2026-08-12 03:58:29,373 [I] DB FLAVOR :POSTGRES
2026-08-12 03:58:29,373 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:58:29,373 [I] Checking connection..
2026-08-12 03:58:29,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 03:58:29,670 [I] Checking connection passed.
2026-08-12 03:58:29,670 [I] --------- Verifying version history table ---------
2026-08-12 03:58:29,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:58:29,946 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:58:29,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:58:30,191 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:58:30,191 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 03:58:30,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 03:58:30.435 UTC [11961] WARNING: there is no transaction in progress
2026-08-12 03:58:30,444 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 03:58:30,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:58:30,720 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 03:58:30,721 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:58:30,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 03:58:31,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 03:58:31,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8tdl.c.gcp-upstream.internal','N') ;"
2026-08-12 03:58:31,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:58:31,835 [I] Table x_portal_user does not exist in database ranger
2026-08-12 03:58:31,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:58:32,088 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 03:58:32,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:58:32,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:58:32,659 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 03:58:32,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 03:58:33.815 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:33.836 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:33.856 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:33.876 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:33.895 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:33.919 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:33.990 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:34.001 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:34.022 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:34.035 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:34.049 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:34.068 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:34.080 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:34.088 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:34.097 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:34.105 UTC [12087] WARNING: there is no transaction in progress
2026-08-12 03:58:34,958 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 03:58:34,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 03:58:35,204 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 03:58:35,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:58:35,477 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 03:58:35,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:58:35,746 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 03:58:35,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:58:36,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:58:36,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8tdl.c.gcp-upstream.internal';"
2026-08-12 03:58:36,583 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 03:58:36,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 03:58:36,866 [I] DB_PATCHES have already been applied
I20260812 03:58:36.872658 31919 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-admin:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskaEzvCg/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 03:58:36.872740 31919 mini_ranger.cc:193] Using host: 127.31.43.212
I20260812 03:58:36.877177 31919 mini_ranger.cc:227] Ranger admin URL: http://127.31.43.212:51067
Aug 12, 2026 3:58:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:58:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:58:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 3:58:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 3:58:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 3:58:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 3:58:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 03:58:38.095104 31919 mini_ranger.cc:160] Time spent starting Ranger: real 9.829s user 0.000s sys 0.013s
Aug 12, 2026 3:59:00 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 3:59:00 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 3:59:00 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 3:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 3:59:00 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 3:59:01 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 03:59:01.895527 31919 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.43.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.43.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/rangerkms_127.31.43.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/rangerkms_127.31.43.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.43.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.43.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.43.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/HTTP_127.31.43.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.43.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/HTTP_127.31.43.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 03:59:01 dist-test-slave-8tdl krb5kdc[11838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507141, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 03:59:01.998409 31919 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 03:59:02.011052 31919 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 03:59:03.083972 31919 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 03:59:03.084060 31919 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-kms
I20260812 03:59:03.084075 31919 mini_ranger_kms.cc:227] Using postgres at 127.31.43.212:46315
2026-08-12 03:59:03,460 [I] DB FLAVOR :POSTGRES
2026-08-12 03:59:03,460 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:59:03,460 [I] Checking connection
2026-08-12 03:59:03,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 03:59:03,743 [I] connection success
2026-08-12 03:59:03,744 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 03:59:03,744 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 03:59:03,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 03:59:04,034 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 03:59:04,034 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:59:04,035 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 03:59:04,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:46315/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 03:59:04,337 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 03:59:04.861140 31919 mini_ranger_kms.cc:310] Created kms service
I20260812 03:59:05.074100 31919 mini_ranger_kms.cc:326] Created kudu user
I20260812 03:59:05.120708 31919 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 03:59:05 dist-test-slave-8tdl krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507141, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 03:59:05.593122 31919 mini_ranger_kms.cc:384] Added ranger policy
I20260812 03:59:05.593256 31919 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-kms:/tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 03:59:05.593281 31919 mini_ranger_kms.cc:240] Using host: 127.31.43.212
I20260812 03:59:05.597635 31919 mini_ranger_kms.cc:276] Ranger KMS PID: 12297
I20260812 03:59:05.597790 31919 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.43.212:42081
Aug 12, 2026 3:59:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:59:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:59:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 3:59:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 3:59:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 3:59:07 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 03:59:08 dist-test-slave-8tdl krb5kdc[11838](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786507148, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.43.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:59:08.061064 31919 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.063s user 0.005s sys 0.348s
I20260812 03:59:08.061208 31919 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 03:59:08.061298 31919 mini_ranger_kms.cc:401] 127.31.43.212:42081/kms/v1/keys
Aug 12 03:59:08 dist-test-slave-8tdl krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507148, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.43.212@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
Aug 12 03:59:08 dist-test-slave-8tdl krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507148, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.43.212@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
Aug 12 03:59:11 dist-test-slave-8tdl krb5kdc[11838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 03:59:11.434542 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40937
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:40937
--ranger_config_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.43.212:42081/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:59:11.524128 12352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:59:11.524569 12352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:59:11.524672 12352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:59:11.526017 12352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:59:11.526103 12352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:59:11.526129 12352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:59:11.526162 12352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:59:11.526188 12352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:59:11.528066 12352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.43.212:42081/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:40937
--ranger_config_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:59:11.528401 12352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:59:11.528658 12352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:59:11.534271 12360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:59:11.534281 12359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:59:11.534399 12352 server_base.cc:1034] running on GCE node
W20260812 03:59:11.534559 12363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:59:11.534901 12352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:59:11.535250 12352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:59:11.536449 12352 hybrid_clock.cc:648] HybridClock initialized: now 1786507151536425 us; error 41 us; skew 500 ppm
Aug 12 03:59:11 dist-test-slave-8tdl krb5kdc[11838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507151, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:59:11.543893 12352 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 03:59:11.544471 12352 webserver.cc:460] Webserver started at http://127.31.43.254:46401/ using document root <none> and password file <none>
I20260812 03:59:11.544739 12352 fs_manager.cc:359] Metadata directory not provided
I20260812 03:59:11.544781 12352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:59:11.544924 12352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:59:11 dist-test-slave-8tdl krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507151, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 03:59:11.595914 12352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "04cdcc0a77d44f0181dcc938bd79d4f6"
format_stamp: "Formatted at 2026-08-12 03:59:11 on dist-test-slave-8tdl"
server_key: "e581395e326b15de633c058d2a996bd8"
server_key_iv: "2d2e8c4394cfdba00d46971ff558171f"
server_key_version: "kuduclusterkey@0"
I20260812 03:59:11.596410 12352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "04cdcc0a77d44f0181dcc938bd79d4f6"
format_stamp: "Formatted at 2026-08-12 03:59:11 on dist-test-slave-8tdl"
server_key: "e581395e326b15de633c058d2a996bd8"
server_key_iv: "2d2e8c4394cfdba00d46971ff558171f"
server_key_version: "kuduclusterkey@0"
I20260812 03:59:11.598091 12352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:59:11.610796 12370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:59:11.611136 12352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:59:11.611282 12352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "04cdcc0a77d44f0181dcc938bd79d4f6"
format_stamp: "Formatted at 2026-08-12 03:59:11 on dist-test-slave-8tdl"
server_key: "e581395e326b15de633c058d2a996bd8"
server_key_iv: "2d2e8c4394cfdba00d46971ff558171f"
server_key_version: "kuduclusterkey@0"
I20260812 03:59:11.611426 12352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:59:11.618963 12352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:59:11.621016 12352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:59:11.621197 12352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:59:11.627045 12352 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:40937
I20260812 03:59:11.627066 12436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:40937 every 8 connection(s)
I20260812 03:59:11.627552 12352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 03:59:11.628412 12437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:59:11.631670 12437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6: Bootstrap starting.
I20260812 03:59:11.632412 12437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6: Neither blocks nor log segments found. Creating new log.
I20260812 03:59:11.632721 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 12352
I20260812 03:59:11.632750 12437 log.cc:824] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6: Log is configured to *not* fsync() on all Append() calls
I20260812 03:59:11.632840 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 03:59:11.633049 31919 external_mini_cluster.cc:1428] Setting key cfab137418413ff449162fa700b341f2
I20260812 03:59:11.633710 12437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6: No bootstrap required, opened a new log
I20260812 03:59:11.636945 12437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cdcc0a77d44f0181dcc938bd79d4f6" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40937 } }
I20260812 03:59:11.637140 12437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:59:11.637176 12437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cdcc0a77d44f0181dcc938bd79d4f6, State: Initialized, Role: FOLLOWER
I20260812 03:59:11.637359 12437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cdcc0a77d44f0181dcc938bd79d4f6" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40937 } }
I20260812 03:59:11.637444 12437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:59:11.637482 12437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:59:11.637547 12437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:59:11.638351 12437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cdcc0a77d44f0181dcc938bd79d4f6" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40937 } }
I20260812 03:59:11.638491 12437 leader_election.cc:302] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04cdcc0a77d44f0181dcc938bd79d4f6; no voters:
I20260812 03:59:11.638852 12437 leader_election.cc:288] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:59:11.638877 12442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:59:11.639178 12442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [term 1 LEADER]: Becoming Leader. State: Replica: 04cdcc0a77d44f0181dcc938bd79d4f6, State: Running, Role: LEADER
I20260812 03:59:11.639286 12437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:59:11.639322 12442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cdcc0a77d44f0181dcc938bd79d4f6" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40937 } }
I20260812 03:59:11.639735 12442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04cdcc0a77d44f0181dcc938bd79d4f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cdcc0a77d44f0181dcc938bd79d4f6" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40937 } } }
I20260812 03:59:11.639959 12442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [sys.catalog]: This master's current role is: LEADER
I20260812 03:59:11.640208 12442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04cdcc0a77d44f0181dcc938bd79d4f6. Latest consensus state: current_term: 1 leader_uuid: "04cdcc0a77d44f0181dcc938bd79d4f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cdcc0a77d44f0181dcc938bd79d4f6" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40937 } } }
I20260812 03:59:11.640363 12442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6 [sys.catalog]: This master's current role is: LEADER
I20260812 03:59:11.640527 12450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:59:11.641255 12450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:59:11 dist-test-slave-8tdl krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:59:11.641820 12437 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:59:11.648138 12450 catalog_manager.cc:1347] Generated new cluster ID: c95b966137cc4c6cba27c51aedc22e7d
I20260812 03:59:11.648296 12450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:59:11.652835 12441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:11.633645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55040 (local address 127.31.43.254:40937)
0812 03:59:11.633852 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 03:59:11.633863 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:59:11.633941 (+ 78us) server_negotiation.cc:407] Connection header received
0812 03:59:11.635306 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:59:11.635317 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:59:11.635391 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:59:11.635568 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:59:11.636386 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:11.637807 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:11.638929 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:11.639566 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:11.639668 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:11.642603 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:59:11.642608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:59:11.642608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:59:11.642719 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:59:11.649599 (+ 6880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:59:11.650108 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:59:11.650111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:59:11.650111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:59:11.650341 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:59:11.650577 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:59:11.650578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:59:11.650578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:59:11.651017 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:59:11.651783 (+ 766us) server_negotiation.cc:1035] Waiting for connection context
0812 03:59:11.651863 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 03:59:11.652181 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":91,"threads_started":1}
I20260812 03:59:11.655894 12450 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:59:11.656662 12450 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:59:11.663729 12450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6: Generated new TSK 0
I20260812 03:59:11.663932 12450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 03:59:12.430908 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.43.212:42081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40937
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:59:12.563863 12486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:59:12.564254 12486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:59:12.564311 12486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:59:12.566171 12486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:59:12.566248 12486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:59:12.566298 12486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 03:59:12.568759 12486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.43.212:42081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:59:12.569077 12486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:59:12.569361 12486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:59:12.575964 12495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:59:12.575927 12492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:59:12.575939 12493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:59:12.576382 12486 server_base.cc:1034] running on GCE node
I20260812 03:59:12.576613 12486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:59:12.576948 12486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:59:12.578146 12486 hybrid_clock.cc:648] HybridClock initialized: now 1786507152578129 us; error 28 us; skew 500 ppm
Aug 12 03:59:12 dist-test-slave-8tdl krb5kdc[11838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:59:12.581751 12486 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 03:59:12.582437 12486 webserver.cc:460] Webserver started at http://127.31.43.193:36975/ using document root <none> and password file <none>
I20260812 03:59:12.582743 12486 fs_manager.cc:359] Metadata directory not provided
I20260812 03:59:12.582814 12486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:59:12.583025 12486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:59:12 dist-test-slave-8tdl krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 03:59:12.599206 12486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "27853f21c11640988931c9ccda08dda2"
format_stamp: "Formatted at 2026-08-12 03:59:12 on dist-test-slave-8tdl"
server_key: "4f907d8b65ba41b019b9a92ec19852d4"
server_key_iv: "d968092c4eee21e2e057c59ee8dd6f16"
server_key_version: "kuduclusterkey@0"
I20260812 03:59:12.599790 12486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "27853f21c11640988931c9ccda08dda2"
format_stamp: "Formatted at 2026-08-12 03:59:12 on dist-test-slave-8tdl"
server_key: "4f907d8b65ba41b019b9a92ec19852d4"
server_key_iv: "d968092c4eee21e2e057c59ee8dd6f16"
server_key_version: "kuduclusterkey@0"
I20260812 03:59:12.605265 12486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260812 03:59:12.623030 12503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:59:12.623266 12486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:59:12.623358 12486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "27853f21c11640988931c9ccda08dda2"
format_stamp: "Formatted at 2026-08-12 03:59:12 on dist-test-slave-8tdl"
server_key: "4f907d8b65ba41b019b9a92ec19852d4"
server_key_iv: "d968092c4eee21e2e057c59ee8dd6f16"
server_key_version: "kuduclusterkey@0"
I20260812 03:59:12.623450 12486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:59:12.636514 12486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:59:12.639578 12486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:59:12.639755 12486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:59:12.640007 12486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:59:12.640417 12486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:59:12.640468 12486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:59:12.640501 12486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:59:12.640519 12486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:59:12.648880 12486 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:39691
I20260812 03:59:12.649272 12631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:39691 every 8 connection(s)
I20260812 03:59:12.649457 12486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:59:12 dist-test-slave-8tdl krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:59:12.662213 12635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:12.650834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:33111 (local address 127.31.43.254:40937)
0812 03:59:12.651047 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 03:59:12.651049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:59:12.651267 (+ 218us) server_negotiation.cc:407] Connection header received
0812 03:59:12.651806 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:59:12.651807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:59:12.651824 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:59:12.651938 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:59:12.652608 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:12.653354 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:12.654546 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:12.654909 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:12.654989 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:12.658111 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:59:12.658113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:59:12.658113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:59:12.658123 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:59:12.660241 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:59:12.660861 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:59:12.660862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:59:12.660863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:59:12.660944 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:59:12.661320 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:59:12.661321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:59:12.661321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:59:12.661497 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:59:12.661574 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:59:12.661973 (+ 399us) server_negotiation.cc:299] Negotiation successful
0812 03:59:12.662085 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":108,"threads_started":1}
I20260812 03:59:12.662714 12636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:12.650831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40937 (local address 127.31.43.193:33111)
0812 03:59:12.651114 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 03:59:12.651167 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:59:12.651695 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:59:12.652022 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:59:12.652025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:59:12.652239 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:59:12.652493 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:12.652497 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:12.653523 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:12.653525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:12.654386 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:12.654389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:12.655088 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:12.655090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:12.655256 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:12.655953 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:59:12.655965 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:59:12.657926 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:59:12.660360 (+ 2434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:59:12.660372 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:59:12.660374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:59:12.660761 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:59:12.661036 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:59:12.661042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:59:12.661042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:59:12.661233 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:59:12.661586 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:59:12.661595 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:59:12.661798 (+ 203us) client_negotiation.cc:769] Sending connection context
0812 03:59:12.661936 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 03:59:12.662143 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":132,"threads_started":1}
I20260812 03:59:12.662931 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 12486
I20260812 03:59:12.663061 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 03:59:12.663247 31919 external_mini_cluster.cc:1428] Setting key 65ba57a14f906b9a33938304ebb278fe
I20260812 03:59:12.663793 12632 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40937
I20260812 03:59:12.663923 12632 heartbeater.cc:464] Registering TS with master...
I20260812 03:59:12.664144 12632 heartbeater.cc:511] Master 127.31.43.254:40937 requested a full tablet report, sending...
I20260812 03:59:12.664732 12393 ts_manager.cc:194] Registered new tserver with Master: 27853f21c11640988931c9ccda08dda2 (127.31.43.193:39691)
I20260812 03:59:12.665684 12393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:33111
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 03:59:12.702736 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.43.212:42081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40937
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:59:12.840494 12640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:59:12.840845 12640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:59:12.840894 12640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:59:12.842918 12640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:59:12.843001 12640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:59:12.843055 12640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 03:59:12.845592 12640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.43.212:42081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:59:12.845927 12640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:59:12.846318 12640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 12 03:59:12 dist-test-slave-8tdl krb5kdc[11838](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786507152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 03:59:12.852905 12650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:59:12.853075 12647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:59:12.852905 12648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:59:12.853386 12640 server_base.cc:1034] running on GCE node
I20260812 03:59:12.853637 12640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:59:12.853962 12640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:59:12.855146 12640 hybrid_clock.cc:648] HybridClock initialized: now 1786507152855117 us; error 37 us; skew 500 ppm
Aug 12 03:59:12 dist-test-slave-8tdl krb5kdc[11838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:59:12.858480 12640 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 03:59:12.859124 12640 webserver.cc:460] Webserver started at http://127.31.43.194:43999/ using document root <none> and password file <none>
I20260812 03:59:12.859385 12640 fs_manager.cc:359] Metadata directory not provided
I20260812 03:59:12.859459 12640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:59:12.859615 12640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:59:12 dist-test-slave-8tdl krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 03:59:12.874189 12640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "6d3ce5485f354aebb9d76d9667a794a7"
format_stamp: "Formatted at 2026-08-12 03:59:12 on dist-test-slave-8tdl"
server_key: "e776d1ded0c4b97ea6dc38a7889dce33"
server_key_iv: "aa7a178fbea32d3f1a14995d2dca83bf"
server_key_version: "kuduclusterkey@0"
I20260812 03:59:12.874850 12640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "6d3ce5485f354aebb9d76d9667a794a7"
format_stamp: "Formatted at 2026-08-12 03:59:12 on dist-test-slave-8tdl"
server_key: "e776d1ded0c4b97ea6dc38a7889dce33"
server_key_iv: "aa7a178fbea32d3f1a14995d2dca83bf"
server_key_version: "kuduclusterkey@0"
I20260812 03:59:12.888222 12640 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.000s sys 0.002s
I20260812 03:59:12.910882 12659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:59:12.911170 12640 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260812 03:59:12.911294 12640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "6d3ce5485f354aebb9d76d9667a794a7"
format_stamp: "Formatted at 2026-08-12 03:59:12 on dist-test-slave-8tdl"
server_key: "e776d1ded0c4b97ea6dc38a7889dce33"
server_key_iv: "aa7a178fbea32d3f1a14995d2dca83bf"
server_key_version: "kuduclusterkey@0"
I20260812 03:59:12.911432 12640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:59:12.923398 12640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:59:12.925977 12640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:59:12.926174 12640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:59:12.926472 12640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:59:12.926934 12640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:59:12.926996 12640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:59:12.927038 12640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:59:12.927062 12640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:59:12.934330 12640 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:43383
I20260812 03:59:12.937357 12782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:43383 every 8 connection(s)
I20260812 03:59:12.937659 12640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 03:59:12.943336 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 12640
I20260812 03:59:12.943478 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 03:59:12.943670 31919 external_mini_cluster.cc:1428] Setting key cd5cfbf4faee93548cf6128da2b7e419
Aug 12 03:59:12 dist-test-slave-8tdl krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:59:12.950923 12635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:12.939400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:50575 (local address 127.31.43.254:40937)
0812 03:59:12.939476 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 03:59:12.939479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:59:12.939656 (+ 177us) server_negotiation.cc:407] Connection header received
0812 03:59:12.940422 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:59:12.940424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:59:12.940444 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:59:12.940510 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:59:12.941363 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:12.942266 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:12.943936 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:12.944272 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:12.944323 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:12.947142 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:59:12.947143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:59:12.947144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:59:12.947152 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:59:12.949076 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:59:12.949658 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:59:12.949660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:59:12.949660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:59:12.949730 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:59:12.950066 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:59:12.950068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:59:12.950068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:59:12.950226 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:59:12.950304 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:59:12.950671 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 03:59:12.950817 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:59:12.951774 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:12.938936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40937 (local address 127.31.43.194:50575)
0812 03:59:12.939505 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 03:59:12.939558 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:59:12.940272 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:59:12.940623 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:59:12.940627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:59:12.940917 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:59:12.941223 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:12.941230 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:12.942396 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:12.942397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:12.943812 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:12.943815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:12.944426 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:12.944427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:12.944578 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:12.945232 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:59:12.945253 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:59:12.947007 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:59:12.949186 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:59:12.949196 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:59:12.949199 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:59:12.949563 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:59:12.949812 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:59:12.949813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:59:12.949813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:59:12.949979 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:59:12.950307 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:59:12.950313 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:59:12.950497 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 03:59:12.951326 (+ 829us) client_negotiation.cc:240] Negotiation successful
0812 03:59:12.951521 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":123,"threads_started":1}
I20260812 03:59:12.952523 12784 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40937
I20260812 03:59:12.952628 12784 heartbeater.cc:464] Registering TS with master...
I20260812 03:59:12.952836 12784 heartbeater.cc:511] Master 127.31.43.254:40937 requested a full tablet report, sending...
I20260812 03:59:12.953325 12393 ts_manager.cc:194] Registered new tserver with Master: 6d3ce5485f354aebb9d76d9667a794a7 (127.31.43.194:43383)
I20260812 03:59:12.953836 12393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:50575
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 03:59:12.991214 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.43.212:42081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40937
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:59:13.092401 12790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:59:13.092665 12790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:59:13.092710 12790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:59:13.094031 12790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:59:13.094112 12790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:59:13.094161 12790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 03:59:13.095863 12790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.43.212:42081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:59:13.096208 12790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:59:13.096596 12790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:59:13.102293 12800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:59:13.102212 12797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:59:13.102212 12796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:59:13.103631 12790 server_base.cc:1034] running on GCE node
I20260812 03:59:13.103880 12790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:59:13.104173 12790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:59:13.105420 12790 hybrid_clock.cc:648] HybridClock initialized: now 1786507153105406 us; error 52 us; skew 500 ppm
Aug 12 03:59:13 dist-test-slave-8tdl krb5kdc[11838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507153, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:59:13.108641 12790 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 03:59:13.109239 12790 webserver.cc:460] Webserver started at http://127.31.43.195:36411/ using document root <none> and password file <none>
I20260812 03:59:13.109529 12790 fs_manager.cc:359] Metadata directory not provided
I20260812 03:59:13.109581 12790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:59:13.109724 12790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:59:13 dist-test-slave-8tdl krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507153, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 03:59:13.123737 12790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "7bc957cc79fd43918384d8770f84525e"
format_stamp: "Formatted at 2026-08-12 03:59:13 on dist-test-slave-8tdl"
server_key: "3e0a857f3fd3478592876bbcd7e9c6e1"
server_key_iv: "66b401731cec76780ef50051caa83d44"
server_key_version: "kuduclusterkey@0"
I20260812 03:59:13.124233 12790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "7bc957cc79fd43918384d8770f84525e"
format_stamp: "Formatted at 2026-08-12 03:59:13 on dist-test-slave-8tdl"
server_key: "3e0a857f3fd3478592876bbcd7e9c6e1"
server_key_iv: "66b401731cec76780ef50051caa83d44"
server_key_version: "kuduclusterkey@0"
I20260812 03:59:13.134404 12790 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.003s sys 0.000s
I20260812 03:59:13.153283 12807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:59:13.158313 12790 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260812 03:59:13.158423 12790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "7bc957cc79fd43918384d8770f84525e"
format_stamp: "Formatted at 2026-08-12 03:59:13 on dist-test-slave-8tdl"
server_key: "3e0a857f3fd3478592876bbcd7e9c6e1"
server_key_iv: "66b401731cec76780ef50051caa83d44"
server_key_version: "kuduclusterkey@0"
I20260812 03:59:13.158550 12790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:59:13.174126 12790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:59:13.176541 12790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:59:13.176736 12790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:59:13.177031 12790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:59:13.177460 12790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:59:13.177517 12790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:59:13.177561 12790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:59:13.177583 12790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:59:13.189628 12790 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:33659
I20260812 03:59:13.190130 12790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 03:59:13.192265 12934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:33659 every 8 connection(s)
I20260812 03:59:13.194139 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 12790
I20260812 03:59:13.194283 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 03:59:13.194535 31919 external_mini_cluster.cc:1428] Setting key 1420af5515f96dafb8ad4196fdc3eccb
Aug 12 03:59:13 dist-test-slave-8tdl krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507153, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 03:59:13.201678 12635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.191797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:35579 (local address 127.31.43.254:40937)
0812 03:59:13.191845 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:59:13.191847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:59:13.191859 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:59:13.192529 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:59:13.192531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:59:13.192543 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:59:13.192605 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:59:13.193230 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.193746 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.195090 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.195448 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.195543 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.198334 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:59:13.198335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:59:13.198335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:59:13.198342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:59:13.199904 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:59:13.200489 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:59:13.200490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:59:13.200491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:59:13.200550 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:59:13.200886 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:59:13.200887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:59:13.200888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:59:13.201007 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:59:13.201100 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:59:13.201503 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 03:59:13.201584 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:59:13.201905 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.191527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40937 (local address 127.31.43.195:35579)
0812 03:59:13.191699 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 03:59:13.191746 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:59:13.192315 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:59:13.192699 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:59:13.192704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:59:13.192910 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:59:13.193129 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:13.193134 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.193854 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.193855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:13.194982 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:13.194986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.195566 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.195567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:13.195718 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.196415 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:59:13.196424 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:59:13.198215 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:59:13.199999 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:59:13.200008 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:59:13.200011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:59:13.200403 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:59:13.200623 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:59:13.200624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:59:13.200624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:59:13.200800 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:59:13.201115 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:59:13.201122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:59:13.201298 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 03:59:13.201489 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 03:59:13.201670 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260812 03:59:13.210501 12935 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40937
I20260812 03:59:13.210736 12935 heartbeater.cc:464] Registering TS with master...
I20260812 03:59:13.210978 12935 heartbeater.cc:511] Master 127.31.43.254:40937 requested a full tablet report, sending...
I20260812 03:59:13.211474 12393 ts_manager.cc:194] Registered new tserver with Master: 7bc957cc79fd43918384d8770f84525e (127.31.43.195:33659)
I20260812 03:59:13.211908 12393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:35579
I20260812 03:59:13.221849 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:59:13.230558 12635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.223517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55048 (local address 127.31.43.254:40937)
0812 03:59:13.223570 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:59:13.223572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:59:13.223709 (+ 137us) server_negotiation.cc:407] Connection header received
0812 03:59:13.223789 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:59:13.223790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:59:13.223801 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:59:13.223875 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:59:13.224506 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.225020 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.225792 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.226002 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.226079 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.227462 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:59:13.227463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:59:13.227464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:59:13.227472 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:59:13.229151 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:59:13.229623 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:59:13.229625 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:59:13.229626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:59:13.229688 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:59:13.229917 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:59:13.229918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:59:13.229919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:59:13.230060 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:59:13.230140 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:59:13.230356 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 03:59:13.230444 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:59:13.232650 12393 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:59:13.234807 12393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:59:13.251447 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.238844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:43383 (local address 127.0.0.1:53764)
0812 03:59:13.246815 (+ 7971us) negotiation.cc:107] Waiting for socket to connect
0812 03:59:13.246893 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 03:59:13.247011 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:59:13.247522 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:59:13.247526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:59:13.247529 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:59:13.247637 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:13.247639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.248636 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.248637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:13.249638 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:13.249640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.250736 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.250737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:13.250824 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.250838 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:59:13.250901 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:59:13.251356 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7915,"thread_start_us":104,"threads_started":1}
I20260812 03:59:13.251863 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.239175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53764 (local address 127.31.43.194:43383)
0812 03:59:13.242946 (+ 3771us) server_negotiation.cc:206] Beginning negotiation
0812 03:59:13.242950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:59:13.246997 (+ 4047us) server_negotiation.cc:407] Connection header received
0812 03:59:13.247140 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:59:13.247143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:59:13.247169 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:59:13.247290 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:59:13.247799 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.248539 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.249761 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.250537 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.250610 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.251695 (+ 1085us) server_negotiation.cc:1035] Waiting for connection context
0812 03:59:13.251725 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 03:59:13.251769 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3735,"thread_start_us":96,"threads_started":1}
I20260812 03:59:13.252336 12951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.239036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:39691 (local address 127.0.0.1:53522)
0812 03:59:13.247283 (+ 8247us) negotiation.cc:107] Waiting for socket to connect
0812 03:59:13.247298 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:59:13.247352 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:59:13.248068 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:59:13.248069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:59:13.248071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:59:13.248129 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:13.248131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.249511 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.249512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:13.250624 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:13.250627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.252090 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.252091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:13.252162 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.252167 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:59:13.252227 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:59:13.252266 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8230,"mutex_wait_us":23,"thread_start_us":64,"threads_started":1}
I20260812 03:59:13.252647 12712 tablet_service.cc:1467] Processing CreateTablet for tablet 58eea2bfa7ce4ac9b945faa171609536 (DEFAULT_TABLE table=test-table [id=138098910994457086016816f7f6b2d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:59:13.252985 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.238857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53522 (local address 127.31.43.193:39691)
0812 03:59:13.242848 (+ 3991us) server_negotiation.cc:206] Beginning negotiation
0812 03:59:13.242853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:59:13.247437 (+ 4584us) server_negotiation.cc:407] Connection header received
0812 03:59:13.247486 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:59:13.247489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:59:13.247507 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:59:13.247580 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:59:13.248222 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.249379 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.250745 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.251846 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.252689 (+ 843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.252765 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:59:13.252796 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 03:59:13.252902 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3941,"thread_start_us":118,"threads_started":1}
I20260812 03:59:13.253101 12712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58eea2bfa7ce4ac9b945faa171609536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:59:13.253291 12953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.239225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:33659 (local address 127.0.0.1:39922)
0812 03:59:13.247693 (+ 8468us) negotiation.cc:107] Waiting for socket to connect
0812 03:59:13.247704 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:59:13.247758 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:59:13.249523 (+ 1765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:59:13.249524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:59:13.249527 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:59:13.249578 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:13.249580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.250874 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.250875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:13.252017 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:13.252020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.253051 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.253052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:13.253121 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.253126 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:59:13.253179 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 03:59:13.253220 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8449,"mutex_wait_us":2,"thread_start_us":62,"threads_started":1}
I20260812 03:59:13.253609 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.239342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39922 (local address 127.31.43.195:33659)
0812 03:59:13.247220 (+ 7878us) server_negotiation.cc:206] Beginning negotiation
0812 03:59:13.247226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:59:13.247825 (+ 599us) server_negotiation.cc:407] Connection header received
0812 03:59:13.247874 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:59:13.247876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:59:13.247894 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:59:13.247969 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:59:13.249671 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.250774 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.252156 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.252871 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.252943 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.253011 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:59:13.253477 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 03:59:13.253528 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7828,"thread_start_us":89,"threads_started":1}
I20260812 03:59:13.254309 12861 tablet_service.cc:1467] Processing CreateTablet for tablet 58eea2bfa7ce4ac9b945faa171609536 (DEFAULT_TABLE table=test-table [id=138098910994457086016816f7f6b2d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:59:13.254781 12861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58eea2bfa7ce4ac9b945faa171609536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:59:13.257849 12560 tablet_service.cc:1467] Processing CreateTablet for tablet 58eea2bfa7ce4ac9b945faa171609536 (DEFAULT_TABLE table=test-table [id=138098910994457086016816f7f6b2d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:59:13.258242 12560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58eea2bfa7ce4ac9b945faa171609536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:59:13.262964 12959 tablet_bootstrap.cc:492] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2: Bootstrap starting.
I20260812 03:59:13.263861 12959 tablet_bootstrap.cc:654] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2: Neither blocks nor log segments found. Creating new log.
I20260812 03:59:13.264345 12959 log.cc:824] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2: Log is configured to *not* fsync() on all Append() calls
I20260812 03:59:13.264832 12956 tablet_bootstrap.cc:492] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7: Bootstrap starting.
I20260812 03:59:13.265970 12956 tablet_bootstrap.cc:654] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7: Neither blocks nor log segments found. Creating new log.
I20260812 03:59:13.266862 12956 log.cc:824] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:59:13.267366 12957 tablet_bootstrap.cc:492] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e: Bootstrap starting.
I20260812 03:59:13.268568 12957 tablet_bootstrap.cc:654] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e: Neither blocks nor log segments found. Creating new log.
I20260812 03:59:13.268911 12957 log.cc:824] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:59:13.269857 12959 tablet_bootstrap.cc:492] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2: No bootstrap required, opened a new log
I20260812 03:59:13.269981 12959 ts_tablet_manager.cc:1397] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260812 03:59:13.272920 12959 raft_consensus.cc:348] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27853f21c11640988931c9ccda08dda2" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 39691 } } peers { permanent_uuid: "7bc957cc79fd43918384d8770f84525e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33659 } } peers { permanent_uuid: "6d3ce5485f354aebb9d76d9667a794a7" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 43383 } }
I20260812 03:59:13.273141 12959 raft_consensus.cc:374] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:59:13.273193 12959 raft_consensus.cc:729] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27853f21c11640988931c9ccda08dda2, State: Initialized, Role: FOLLOWER
I20260812 03:59:13.273738 12956 tablet_bootstrap.cc:492] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7: No bootstrap required, opened a new log
I20260812 03:59:13.273835 12956 ts_tablet_manager.cc:1397] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20260812 03:59:13.275779 12957 tablet_bootstrap.cc:492] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e: No bootstrap required, opened a new log
I20260812 03:59:13.275923 12957 ts_tablet_manager.cc:1397] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260812 03:59:13.276135 12956 raft_consensus.cc:348] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27853f21c11640988931c9ccda08dda2" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 39691 } } peers { permanent_uuid: "7bc957cc79fd43918384d8770f84525e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33659 } } peers { permanent_uuid: "6d3ce5485f354aebb9d76d9667a794a7" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 43383 } }
I20260812 03:59:13.276332 12956 raft_consensus.cc:374] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:59:13.276381 12956 raft_consensus.cc:729] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d3ce5485f354aebb9d76d9667a794a7, State: Initialized, Role: FOLLOWER
I20260812 03:59:13.276551 12956 consensus_queue.cc:260] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27853f21c11640988931c9ccda08dda2" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 39691 } } peers { permanent_uuid: "7bc957cc79fd43918384d8770f84525e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33659 } } peers { permanent_uuid: "6d3ce5485f354aebb9d76d9667a794a7" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 43383 } }
I20260812 03:59:13.276417 12959 consensus_queue.cc:260] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27853f21c11640988931c9ccda08dda2" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 39691 } } peers { permanent_uuid: "7bc957cc79fd43918384d8770f84525e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33659 } } peers { permanent_uuid: "6d3ce5485f354aebb9d76d9667a794a7" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 43383 } }
I20260812 03:59:13.276880 12956 ts_tablet_manager.cc:1428] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:59:13.277033 12784 heartbeater.cc:503] Master 127.31.43.254:40937 was elected leader, sending a full tablet report...
I20260812 03:59:13.277359 12959 ts_tablet_manager.cc:1428] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2: Time spent starting tablet: real 0.007s user 0.000s sys 0.004s
I20260812 03:59:13.278178 12632 heartbeater.cc:503] Master 127.31.43.254:40937 was elected leader, sending a full tablet report...
I20260812 03:59:13.279695 12957 raft_consensus.cc:348] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27853f21c11640988931c9ccda08dda2" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 39691 } } peers { permanent_uuid: "7bc957cc79fd43918384d8770f84525e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33659 } } peers { permanent_uuid: "6d3ce5485f354aebb9d76d9667a794a7" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 43383 } }
I20260812 03:59:13.279896 12957 raft_consensus.cc:374] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:59:13.279937 12957 raft_consensus.cc:729] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc957cc79fd43918384d8770f84525e, State: Initialized, Role: FOLLOWER
I20260812 03:59:13.280071 12957 consensus_queue.cc:260] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27853f21c11640988931c9ccda08dda2" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 39691 } } peers { permanent_uuid: "7bc957cc79fd43918384d8770f84525e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33659 } } peers { permanent_uuid: "6d3ce5485f354aebb9d76d9667a794a7" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 43383 } }
I20260812 03:59:13.280481 12957 ts_tablet_manager.cc:1428] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 03:59:13.280534 12935 heartbeater.cc:503] Master 127.31.43.254:40937 was elected leader, sending a full tablet report...
Aug 12 03:59:13 dist-test-slave-8tdl krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
W20260812 03:59:13.401089 12633 tablet.cc:2369] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:59:13.417168 12964 raft_consensus.cc:482] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:59:13.417353 12964 raft_consensus.cc:504] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27853f21c11640988931c9ccda08dda2" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 39691 } } peers { permanent_uuid: "7bc957cc79fd43918384d8770f84525e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33659 } } peers { permanent_uuid: "6d3ce5485f354aebb9d76d9667a794a7" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 43383 } }
I20260812 03:59:13.417809 12964 leader_election.cc:288] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27853f21c11640988931c9ccda08dda2 (127.31.43.193:39691), 7bc957cc79fd43918384d8770f84525e (127.31.43.195:33659)
I20260812 03:59:13.422067 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.418027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:39691 (local address 127.31.43.194:58053)
0812 03:59:13.418081 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 03:59:13.418096 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:59:13.418152 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:59:13.418593 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:59:13.418594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:59:13.418596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:59:13.418712 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:13.418714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.419670 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.419672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:13.420921 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:13.420925 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.421775 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.421776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:13.421867 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.421871 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:59:13.421922 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:59:13.421966 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 03:59:13.422618 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.418356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:58053 (local address 127.31.43.193:39691)
0812 03:59:13.418425 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:59:13.418429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:59:13.418439 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:59:13.418471 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:59:13.418472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:59:13.418481 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:59:13.418535 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:59:13.418803 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.419563 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.421054 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.421652 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.421728 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.421739 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:59:13.422495 (+ 756us) server_negotiation.cc:299] Negotiation successful
0812 03:59:13.422550 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:59:13.422942 12581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58eea2bfa7ce4ac9b945faa171609536" candidate_uuid: "6d3ce5485f354aebb9d76d9667a794a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27853f21c11640988931c9ccda08dda2" is_pre_election: true
I20260812 03:59:13.423151 12581 raft_consensus.cc:2454] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d3ce5485f354aebb9d76d9667a794a7 in term 0.
I20260812 03:59:13.423434 12663 leader_election.cc:302] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27853f21c11640988931c9ccda08dda2, 6d3ce5485f354aebb9d76d9667a794a7; no voters:
I20260812 03:59:13.423580 12964 raft_consensus.cc:2781] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:59:13.423666 12964 raft_consensus.cc:482] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:59:13.423694 12964 raft_consensus.cc:3037] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:59:13.424624 12964 raft_consensus.cc:504] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27853f21c11640988931c9ccda08dda2" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 39691 } } peers { permanent_uuid: "7bc957cc79fd43918384d8770f84525e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33659 } } peers { permanent_uuid: "6d3ce5485f354aebb9d76d9667a794a7" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 43383 } }
I20260812 03:59:13.424782 12964 leader_election.cc:288] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [CANDIDATE]: Term 1 election: Requested vote from peers 27853f21c11640988931c9ccda08dda2 (127.31.43.193:39691), 7bc957cc79fd43918384d8770f84525e (127.31.43.195:33659)
I20260812 03:59:13.425020 12581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58eea2bfa7ce4ac9b945faa171609536" candidate_uuid: "6d3ce5485f354aebb9d76d9667a794a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27853f21c11640988931c9ccda08dda2"
I20260812 03:59:13.425113 12581 raft_consensus.cc:3037] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:59:13.426013 12581 raft_consensus.cc:2454] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d3ce5485f354aebb9d76d9667a794a7 in term 1.
I20260812 03:59:13.426210 12663 leader_election.cc:302] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27853f21c11640988931c9ccda08dda2, 6d3ce5485f354aebb9d76d9667a794a7; no voters:
I20260812 03:59:13.426302 12964 raft_consensus.cc:2781] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:59:13.426477 12964 raft_consensus.cc:686] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 1 LEADER]: Becoming Leader. State: Replica: 6d3ce5485f354aebb9d76d9667a794a7, State: Running, Role: LEADER
I20260812 03:59:13.426585 12964 consensus_queue.cc:237] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27853f21c11640988931c9ccda08dda2" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 39691 } } peers { permanent_uuid: "7bc957cc79fd43918384d8770f84525e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33659 } } peers { permanent_uuid: "6d3ce5485f354aebb9d76d9667a794a7" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 43383 } }
I20260812 03:59:13.427516 12393 catalog_manager.cc:5495] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d3ce5485f354aebb9d76d9667a794a7 (127.31.43.194). New cstate: current_term: 1 leader_uuid: "6d3ce5485f354aebb9d76d9667a794a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27853f21c11640988931c9ccda08dda2" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 39691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc957cc79fd43918384d8770f84525e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d3ce5485f354aebb9d76d9667a794a7" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 43383 } health_report { overall_health: HEALTHY } } }
I20260812 03:59:13.434357 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.418420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:40607 (local address 127.31.43.195:33659)
0812 03:59:13.418476 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:59:13.418479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:59:13.418488 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:59:13.418517 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:59:13.418518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:59:13.418529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:59:13.418585 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:59:13.431007 (+ 12422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.431829 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.433307 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.433887 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.433944 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.433953 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:59:13.434221 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 03:59:13.434270 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:59:13.434667 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.418052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:33659 (local address 127.31.43.194:40607)
0812 03:59:13.418226 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 03:59:13.418238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:59:13.418297 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:59:13.430742 (+ 12445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:59:13.430744 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:59:13.430747 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:59:13.430861 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:13.430865 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.431987 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.431988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:13.433182 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:59:13.433185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.434045 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.434046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:59:13.434118 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.434123 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:59:13.434510 (+ 387us) client_negotiation.cc:240] Negotiation successful
0812 03:59:13.434568 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":111,"threads_started":1}
I20260812 03:59:13.435137 12870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58eea2bfa7ce4ac9b945faa171609536" candidate_uuid: "6d3ce5485f354aebb9d76d9667a794a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc957cc79fd43918384d8770f84525e" is_pre_election: true
I20260812 03:59:13.435110 12869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58eea2bfa7ce4ac9b945faa171609536" candidate_uuid: "6d3ce5485f354aebb9d76d9667a794a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc957cc79fd43918384d8770f84525e"
I20260812 03:59:13.435328 12870 raft_consensus.cc:2454] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d3ce5485f354aebb9d76d9667a794a7 in term 0.
W20260812 03:59:13.438803 12785 tablet.cc:2369] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:59:13.442814 12936 tablet.cc:2369] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:59:13.470234 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:59:13.466968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53780 (local address 127.31.43.194:43383)
0812 03:59:13.467021 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:59:13.467024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:59:13.467034 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:59:13.467083 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:59:13.467083 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:59:13.467108 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:59:13.467186 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:59:13.467411 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.468084 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.469153 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:59:13.469415 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:59:13.469481 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:59:13.469694 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:59:13.469837 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:59:13.469918 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:59:13.470126 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 03:59:13.470153 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:59:13.473657 12870 raft_consensus.cc:3037] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:59:13.473740 12581 raft_consensus.cc:1261] T 58eea2bfa7ce4ac9b945faa171609536 P 27853f21c11640988931c9ccda08dda2 [term 1 FOLLOWER]: Refusing update from remote peer 6d3ce5485f354aebb9d76d9667a794a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:59:13.474165 12964 consensus_queue.cc:1035] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27853f21c11640988931c9ccda08dda2" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 39691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:59:13.474967 12870 raft_consensus.cc:1261] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e [term 1 FOLLOWER]: Refusing update from remote peer 6d3ce5485f354aebb9d76d9667a794a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:59:13.475239 12964 consensus_queue.cc:1035] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc957cc79fd43918384d8770f84525e" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:59:13.479493 12973 mvcc.cc:205] Tried to move back new op lower bound from 7317533300625100800 to 7317533300436168704. Current Snapshot: MvccSnapshot[applied={T|T < 7317533300625100800}]
I20260812 03:59:13.479712 12976 mvcc.cc:205] Tried to move back new op lower bound from 7317533300625100800 to 7317533300436168704. Current Snapshot: MvccSnapshot[applied={T|T < 7317533300625100800}]
I20260812 03:59:13.482240 12978 mvcc.cc:205] Tried to move back new op lower bound from 7317533300625100800 to 7317533300436168704. Current Snapshot: MvccSnapshot[applied={T|T < 7317533300625100800}]
I20260812 03:59:13.483158 12392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:59:13.483273 12392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:59:13.484045 12392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04cdcc0a77d44f0181dcc938bd79d4f6: Sending DeleteTablet for 3 replicas of tablet 58eea2bfa7ce4ac9b945faa171609536
I20260812 03:59:13.484438 12560 tablet_service.cc:1514] Processing DeleteTablet for tablet 58eea2bfa7ce4ac9b945faa171609536 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:59:13 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:53522
I20260812 03:59:13.484961 12861 tablet_service.cc:1514] Processing DeleteTablet for tablet 58eea2bfa7ce4ac9b945faa171609536 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:59:13 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:39922
I20260812 03:59:13.485518 12712 tablet_service.cc:1514] Processing DeleteTablet for tablet 58eea2bfa7ce4ac9b945faa171609536 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:59:13 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:53764
I20260812 03:59:13.486096 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 12486
I20260812 03:59:13.491822 12986 tablet_replica.cc:330] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7: stopping tablet replica
I20260812 03:59:13.491993 12986 raft_consensus.cc:2229] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:59:13.492110 12986 raft_consensus.cc:2258] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:59:13.492301 12985 tablet_replica.cc:330] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e: stopping tablet replica
I20260812 03:59:13.492443 12985 raft_consensus.cc:2229] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:59:13.492539 12985 raft_consensus.cc:2258] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 03:59:13.498358 12377 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:39691 (error 108)
I20260812 03:59:13.499477 12986 ts_tablet_manager.cc:1905] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:59:13.501762 12985 ts_tablet_manager.cc:1905] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:59:13.503758 12986 ts_tablet_manager.cc:1918] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:59:13.503876 12986 log.cc:1196] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/58eea2bfa7ce4ac9b945faa171609536
I20260812 03:59:13.504155 12986 ts_tablet_manager.cc:1939] T 58eea2bfa7ce4ac9b945faa171609536 P 6d3ce5485f354aebb9d76d9667a794a7: Deleting consensus metadata
I20260812 03:59:13.504634 12985 ts_tablet_manager.cc:1918] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:59:13.504721 12985 log.cc:1196] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/58eea2bfa7ce4ac9b945faa171609536
I20260812 03:59:13.504731 12372 catalog_manager.cc:4841] TS 6d3ce5485f354aebb9d76d9667a794a7 (127.31.43.194:43383): tablet 58eea2bfa7ce4ac9b945faa171609536 (table test-table [id=138098910994457086016816f7f6b2d7]) successfully deleted
I20260812 03:59:13.505014 12985 ts_tablet_manager.cc:1939] T 58eea2bfa7ce4ac9b945faa171609536 P 7bc957cc79fd43918384d8770f84525e: Deleting consensus metadata
W20260812 03:59:13.505399 12953 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:59:13.505215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:39691 (local address 127.0.0.1:53538)
0812 03:59:13.505276 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 03:59:13.505292 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:59:13.505353 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:39691: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 03:59:13.505539 12377 catalog_manager.cc:4584] TS 27853f21c11640988931c9ccda08dda2 (127.31.43.193:39691): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 58eea2bfa7ce4ac9b945faa171609536: Network error: Client connection negotiation failed: client connection to 127.31.43.193:39691: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260812 03:59:13.507800 12375 catalog_manager.cc:4841] TS 7bc957cc79fd43918384d8770f84525e (127.31.43.195:33659): tablet 58eea2bfa7ce4ac9b945faa171609536 (table test-table [id=138098910994457086016816f7f6b2d7]) successfully deleted
I20260812 03:59:13.508093 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 12640
I20260812 03:59:13.519098 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 12790
I20260812 03:59:13.528265 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 12352
I20260812 03:59:13.541131 31919 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 03:59:13.886679 31919 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 03:59:13.886861 31919 mini_ranger.cc:66] Stopping Ranger...
I20260812 03:59:14.243384 31919 mini_ranger.cc:68] Stopped Ranger
2026-08-12 03:59:14.243 UTC [11872] LOG: received smart shutdown request
2026-08-12 03:59:14.248 UTC [11872] LOG: background worker "logical replication launcher" (PID 11883) exited with exit code 1
2026-08-12 03:59:14.249 UTC [11878] LOG: shutting down
2026-08-12 03:59:14.479 UTC [11872] LOG: database system is shut down
2026-08-12T03:59:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51848 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:59:14 dist-test-slave-8tdl krb5kdc[12999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:59:14 dist-test-slave-8tdl krb5kdc[12999](info): set up 2 sockets
Aug 12 03:59:14 dist-test-slave-8tdl krb5kdc[12999](info): commencing operation
krb5kdc: starting...
W20260812 03:59:15.748224 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 03:59:15 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:59:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:59:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.43.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.43.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/rangeradmin_127.31.43.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/rangeradmin_127.31.43.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.43.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.43.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/rangerlookup_127.31.43.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/rangerlookup_127.31.43.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.43.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.43.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/HTTP_127.31.43.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/HTTP_127.31.43.212@KRBTEST.COM.keytab.
I20260812 03:59:15.950394 31919 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskaEzvCg/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/postgres -l logfile start
2026-08-12 03:59:18.932 UTC [13032] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 03:59:18.932 UTC [13032] LOG: listening on IPv4 address "127.31.43.212", port 55109
2026-08-12 03:59:18.937 UTC [13032] LOG: listening on Unix socket "/tmp/.s.PGSQL.55109"
2026-08-12 03:59:18.975 UTC [13035] LOG: database system was shut down at 2026-08-12 03:59:17 UTC
2026-08-12 03:59:18.979 UTC [13032] LOG: database system is ready to accept connections
I20260812 03:59:20.010746 31919 mini_postgres.cc:96] Postgres bound to 55109
2026-08-12 03:59:20.018 UTC [13045] FATAL: database "slave" does not exist
127.31.43.212:55109 - accepting connections
I20260812 03:59:20.019237 31919 mini_ranger.cc:161] Starting Ranger...
I20260812 03:59:20.037041 31919 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 03:59:20.619902 31919 mini_ranger.cc:87] Created ranger Postgres database
I20260812 03:59:20.620018 31919 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-admin
2026-08-12 03:59:20,896 [I] DB FLAVOR :POSTGRES
2026-08-12 03:59:20,896 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:59:20,897 [I] Checking connection..
2026-08-12 03:59:20,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 03:59:21,145 [I] Checking connection passed.
2026-08-12 03:59:21,145 [I] --------- Verifying version history table ---------
2026-08-12 03:59:21,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:59:21,405 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:59:21,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:59:21,655 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:59:21,655 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 03:59:21,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 03:59:21.910 UTC [13121] WARNING: there is no transaction in progress
2026-08-12 03:59:21,918 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 03:59:21,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:59:22,202 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 03:59:22,202 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:59:22,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 03:59:22,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 03:59:22,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8tdl.c.gcp-upstream.internal','N') ;"
2026-08-12 03:59:22,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:59:23,233 [I] Table x_portal_user does not exist in database ranger
2026-08-12 03:59:23,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:59:23,500 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 03:59:23,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:59:23,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:59:24,012 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 03:59:24,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 03:59:25.112 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.132 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.153 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.171 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.189 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.209 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.273 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.281 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.298 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.311 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.324 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.341 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.353 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.362 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.370 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:25.378 UTC [13245] WARNING: there is no transaction in progress
2026-08-12 03:59:26,171 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 03:59:26,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 03:59:26,454 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 03:59:26,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:59:26,722 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 03:59:26,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:59:26,988 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 03:59:26,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:59:27,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:59:27,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8tdl.c.gcp-upstream.internal';"
2026-08-12 03:59:27,785 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 03:59:27,786 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 03:59:28,054 [I] DB_PATCHES have already been applied
I20260812 03:59:28.060530 31919 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-admin:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskaEzvCg/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 03:59:28.060636 31919 mini_ranger.cc:193] Using host: 127.31.43.212
I20260812 03:59:28.064970 31919 mini_ranger.cc:227] Ranger admin URL: http://127.31.43.212:57601
Aug 12, 2026 3:59:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:59:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:59:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 3:59:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 3:59:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 3:59:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 3:59:29 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 03:59:29.268035 31919 mini_ranger.cc:160] Time spent starting Ranger: real 9.249s user 0.000s sys 0.012s
Aug 12, 2026 3:59:49 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 3:59:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 3:59:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 3:59:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 3:59:49 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 3:59:50 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 03:59:51.660028 31919 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.43.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.43.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/rangerkms_127.31.43.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.43.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/rangerkms_127.31.43.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.43.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.43.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.43.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/HTTP_127.31.43.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.43.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/HTTP_127.31.43.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 03:59:51 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507191, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 03:59:51.777446 31919 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 03:59:51.789274 31919 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 03:59:52.841667 31919 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 03:59:52.841780 31919 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-kms
I20260812 03:59:52.841804 31919 mini_ranger_kms.cc:227] Using postgres at 127.31.43.212:55109
2026-08-12 03:59:53,026 [I] DB FLAVOR :POSTGRES
2026-08-12 03:59:53,027 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:59:53,027 [I] Checking connection
2026-08-12 03:59:53,027 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 03:59:53,303 [I] connection success
2026-08-12 03:59:53,303 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 03:59:53,303 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 03:59:53,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 03:59:53,560 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 03:59:53,560 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:59:53,560 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 03:59:53,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.43.212:55109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 03:59:53,870 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 03:59:54.405071 31919 mini_ranger_kms.cc:310] Created kms service
I20260812 03:59:54.660692 31919 mini_ranger_kms.cc:326] Created kudu user
I20260812 03:59:54.710269 31919 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 03:59:54 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507191, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 03:59:55.208361 31919 mini_ranger_kms.cc:384] Added ranger policy
I20260812 03:59:55.208626 31919 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-kms:/tmp/dist-test-taskaEzvCg/build/release/bin/postgresql.jar:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskaEzvCg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 03:59:55.208659 31919 mini_ranger_kms.cc:240] Using host: 127.31.43.212
I20260812 03:59:55.213034 31919 mini_ranger_kms.cc:276] Ranger KMS PID: 13462
I20260812 03:59:55.213126 31919 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.43.212:42663
Aug 12, 2026 3:59:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:59:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:59:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 3:59:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 3:59:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 3:59:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 03:59:57 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786507197, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.43.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:59:57.723631 31919 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.946s user 0.004s sys 0.164s
I20260812 03:59:57.723879 31919 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 03:59:57.723999 31919 mini_ranger_kms.cc:401] 127.31.43.212:42663/kms/v1/keys
Aug 12 03:59:58 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507197, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.43.212@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
Aug 12 03:59:58 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507197, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.43.212@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
Aug 12 04:00:00 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:00:00.878906 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:37343
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:37343
--ranger_config_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.43.212:42663/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:00.967582 13521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:00.967808 13521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:00.967830 13521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:00.969102 13521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:00:00.969151 13521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:00.969164 13521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:00.969177 13521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:00:00.969189 13521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:00:00.970960 13521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.43.212:42663/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:37343
--ranger_config_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:37343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:00.971238 13521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:00.971484 13521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:00.978508 13528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:00.978551 13521 server_base.cc:1034] running on GCE node
W20260812 04:00:00.978641 13530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:00.978503 13527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:00.979007 13521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:00.979405 13521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:00.980583 13521 hybrid_clock.cc:648] HybridClock initialized: now 1786507200980556 us; error 43 us; skew 500 ppm
Aug 12 04:00:00 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507200, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:00.993902 13521 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:00:00.994560 13521 webserver.cc:460] Webserver started at http://127.31.43.254:46703/ using document root <none> and password file <none>
I20260812 04:00:00.994885 13521 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:00.994954 13521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:00.995127 13521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 04:00:00 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507200, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 04:00:01.046059 13521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "ab9c9ed359784d2182a02e4d6678c286"
format_stamp: "Formatted at 2026-08-12 04:00:01 on dist-test-slave-8tdl"
server_key: "dae951df93e9d0e420189db18f319369"
server_key_iv: "a2acfd60f14aa6728cc0c8ceddd9455b"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:01.046572 13521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "ab9c9ed359784d2182a02e4d6678c286"
format_stamp: "Formatted at 2026-08-12 04:00:01 on dist-test-slave-8tdl"
server_key: "dae951df93e9d0e420189db18f319369"
server_key_iv: "a2acfd60f14aa6728cc0c8ceddd9455b"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:01.048357 13521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:00:01.060791 13537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:01.061024 13521 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:00:01.061153 13521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "ab9c9ed359784d2182a02e4d6678c286"
format_stamp: "Formatted at 2026-08-12 04:00:01 on dist-test-slave-8tdl"
server_key: "dae951df93e9d0e420189db18f319369"
server_key_iv: "a2acfd60f14aa6728cc0c8ceddd9455b"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:01.061259 13521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:01.101480 13521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:01.103896 13521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:01.104075 13521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:01.109360 13521 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:37343
I20260812 04:00:01.109439 13594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:37343 every 8 connection(s)
I20260812 04:00:01.109874 13521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:00:01.110751 13596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:01.113303 13596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286: Bootstrap starting.
I20260812 04:00:01.113976 13596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:01.114305 13596 log.cc:824] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:01.115252 13596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286: No bootstrap required, opened a new log
I20260812 04:00:01.117286 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 13521
I20260812 04:00:01.117406 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:00:01.117597 31919 external_mini_cluster.cc:1428] Setting key f0c37bf5b9c3face0a32b79ba51bb943
I20260812 04:00:01.117579 13596 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9c9ed359784d2182a02e4d6678c286" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37343 } }
I20260812 04:00:01.117811 13596 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:01.117887 13596 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab9c9ed359784d2182a02e4d6678c286, State: Initialized, Role: FOLLOWER
I20260812 04:00:01.118278 13596 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9c9ed359784d2182a02e4d6678c286" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37343 } }
I20260812 04:00:01.118396 13596 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:00:01.118435 13596 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:00:01.118561 13596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:01.119350 13596 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9c9ed359784d2182a02e4d6678c286" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37343 } }
I20260812 04:00:01.119482 13596 leader_election.cc:302] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab9c9ed359784d2182a02e4d6678c286; no voters:
I20260812 04:00:01.119712 13596 leader_election.cc:288] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:00:01.119992 13603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:00:01.120057 13596 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:00:01.120224 13603 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 1 LEADER]: Becoming Leader. State: Replica: ab9c9ed359784d2182a02e4d6678c286, State: Running, Role: LEADER
I20260812 04:00:01.120411 13603 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9c9ed359784d2182a02e4d6678c286" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37343 } }
I20260812 04:00:01.120821 13603 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab9c9ed359784d2182a02e4d6678c286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9c9ed359784d2182a02e4d6678c286" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37343 } } }
I20260812 04:00:01.120982 13603 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:01.121166 13603 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab9c9ed359784d2182a02e4d6678c286. Latest consensus state: current_term: 1 leader_uuid: "ab9c9ed359784d2182a02e4d6678c286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9c9ed359784d2182a02e4d6678c286" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37343 } } }
I20260812 04:00:01.121277 13603 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:01.121438 13613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:00:01.121695 13596 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 04:00:01.125890 13613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 04:00:01 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:01.134371 13600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:01.118168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50694 (local address 127.31.43.254:37343)
0812 04:00:01.118524 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:01.118535 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:01.118568 (+ 33us) server_negotiation.cc:407] Connection header received
0812 04:00:01.118728 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:01.118736 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:01.118799 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:01.118945 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:01.119769 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:01.121449 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:01.122592 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:01.125892 (+ 3300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:01.126118 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:01.129680 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:01.129684 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:01.129685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:01.129699 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:01.131949 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:01.132408 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:01.132412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:01.132412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:01.132537 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:01.132754 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:01.132755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:01.132755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:01.133126 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:01.133274 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:01.133959 (+ 685us) server_negotiation.cc:299] Negotiation successful
0812 04:00:01.134113 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":106,"threads_started":1}
I20260812 04:00:01.139289 13613 catalog_manager.cc:1347] Generated new cluster ID: efa30d77c34a4761b4d212f596484626
I20260812 04:00:01.139364 13613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:00:01.160403 13613 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:00:01.161226 13613 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:00:01.166328 13613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286: Generated new TSK 0
I20260812 04:00:01.166661 13613 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 04:00:01.959383 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.43.212:42663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:37343
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:02.076257 13650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:02.076630 13650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:02.076722 13650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:02.078619 13650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:02.078751 13650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:02.078820 13650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:00:02.081139 13650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.43.212:42663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:37343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:02.081419 13650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:02.081738 13650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:02.088559 13655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:02.088680 13657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:02.088570 13659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:02.089249 13650 server_base.cc:1034] running on GCE node
I20260812 04:00:02.089479 13650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:02.089766 13650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:02.090917 13650 hybrid_clock.cc:648] HybridClock initialized: now 1786507202090899 us; error 40 us; skew 500 ppm
Aug 12 04:00:02 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507202, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:02.093900 13650 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 04:00:02.094381 13650 webserver.cc:460] Webserver started at http://127.31.43.193:34111/ using document root <none> and password file <none>
I20260812 04:00:02.094662 13650 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:02.094790 13650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:02.094967 13650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 04:00:02 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507202, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 04:00:02.110848 13650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "a2c43343d0e047138e6d6f4faf8616ea"
format_stamp: "Formatted at 2026-08-12 04:00:02 on dist-test-slave-8tdl"
server_key: "27e26230d74368a70543f988788568e6"
server_key_iv: "f33ec52ccf1a0ede246ccc623545c324"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:02.111413 13650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "a2c43343d0e047138e6d6f4faf8616ea"
format_stamp: "Formatted at 2026-08-12 04:00:02 on dist-test-slave-8tdl"
server_key: "27e26230d74368a70543f988788568e6"
server_key_iv: "f33ec52ccf1a0ede246ccc623545c324"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:02.115911 13650 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260812 04:00:02.131673 13667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:02.135159 13650 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260812 04:00:02.135406 13650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "a2c43343d0e047138e6d6f4faf8616ea"
format_stamp: "Formatted at 2026-08-12 04:00:02 on dist-test-slave-8tdl"
server_key: "27e26230d74368a70543f988788568e6"
server_key_iv: "f33ec52ccf1a0ede246ccc623545c324"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:02.135655 13650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:02.166870 13650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:02.169126 13650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:02.169301 13650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:02.169588 13650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:02.169983 13650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:02.170043 13650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:02.170111 13650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:02.170156 13650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:02.178069 13650 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:38315
I20260812 04:00:02.178217 13795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:38315 every 8 connection(s)
I20260812 04:00:02.178539 13650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
Aug 12 04:00:02 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507202, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:02.188145 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 13650
I20260812 04:00:02.188267 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 04:00:02.188478 31919 external_mini_cluster.cc:1428] Setting key 0dc8481afd69428d2f69d3a252af42cc
I20260812 04:00:02.188639 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:02.179625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:55053 (local address 127.31.43.254:37343)
0812 04:00:02.179739 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:02.179741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:02.179753 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:00:02.180296 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:02.180297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:02.180314 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:02.180374 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:02.180947 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:02.181469 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:02.182453 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:02.182715 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:02.182787 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:02.185274 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:02.185275 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:02.185276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:02.185283 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:02.187163 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:02.187592 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:02.187593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:02.187593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:02.187663 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:02.187933 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:02.187933 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:02.187934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:02.188063 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:02.188145 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:02.188446 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 04:00:02.188538 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":50,"threads_started":1}
I20260812 04:00:02.188809 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:02.179375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37343 (local address 127.31.43.193:55053)
0812 04:00:02.179554 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:02.179599 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:02.180175 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:02.180444 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:02.180448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:02.180650 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:02.180846 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:02.180850 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:02.181570 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:02.181571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:02.182358 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:02.182361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:02.182812 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:02.182813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:02.182934 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:02.183505 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:02.183513 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:02.185175 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:02.187254 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:02.187261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:02.187264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:02.187496 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:02.187732 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:02.187732 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:02.187733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:02.187847 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:02.188134 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:02.188141 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:02.188302 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 04:00:02.188408 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 04:00:02.188566 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260812 04:00:02.189369 13796 heartbeater.cc:347] Connected to a master server at 127.31.43.254:37343
I20260812 04:00:02.189502 13796 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:02.189690 13796 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
I20260812 04:00:02.190263 13555 ts_manager.cc:194] Registered new tserver with Master: a2c43343d0e047138e6d6f4faf8616ea (127.31.43.193:38315)
I20260812 04:00:02.190960 13555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:55053
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 04:00:02.227110 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.43.212:42663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:37343
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 04:00:02 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786507202, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 04:00:02.345786 13804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:02.346200 13804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:02.346288 13804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:02.348063 13804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:02.348183 13804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:02.348269 13804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:00:02.350188 13804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.43.212:42663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:37343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:02.350459 13804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:02.350796 13804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:02.357396 13812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:02.358319 13811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:02.358603 13804 server_base.cc:1034] running on GCE node
W20260812 04:00:02.358350 13815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:02.358937 13804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:02.359246 13804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:02.360412 13804 hybrid_clock.cc:648] HybridClock initialized: now 1786507202360389 us; error 37 us; skew 500 ppm
Aug 12 04:00:02 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507202, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:02.363447 13804 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 04:00:02.363973 13804 webserver.cc:460] Webserver started at http://127.31.43.194:38433/ using document root <none> and password file <none>
I20260812 04:00:02.364261 13804 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:02.364321 13804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:02.364459 13804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 04:00:02 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507202, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 04:00:02.379606 13804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "88109a58f1da4fe095ea1bdd424b2ba4"
format_stamp: "Formatted at 2026-08-12 04:00:02 on dist-test-slave-8tdl"
server_key: "e6c9be3827b4f83096e280e5e3a352c0"
server_key_iv: "6054ada7714a33a2b7a57d9a28a2a486"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:02.380374 13804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "88109a58f1da4fe095ea1bdd424b2ba4"
format_stamp: "Formatted at 2026-08-12 04:00:02 on dist-test-slave-8tdl"
server_key: "e6c9be3827b4f83096e280e5e3a352c0"
server_key_iv: "6054ada7714a33a2b7a57d9a28a2a486"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:02.383136 13804 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 04:00:02.400614 13822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:02.400843 13804 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260812 04:00:02.400964 13804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "88109a58f1da4fe095ea1bdd424b2ba4"
format_stamp: "Formatted at 2026-08-12 04:00:02 on dist-test-slave-8tdl"
server_key: "e6c9be3827b4f83096e280e5e3a352c0"
server_key_iv: "6054ada7714a33a2b7a57d9a28a2a486"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:02.401101 13804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:02.419930 13804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:02.422928 13804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:02.423118 13804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:02.423331 13804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:02.423839 13804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:02.423899 13804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:02.423935 13804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:02.423956 13804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:02.431458 13804 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:34679
I20260812 04:00:02.432022 13804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:00:02.436295 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 13804
I20260812 04:00:02.436424 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 04:00:02.436638 31919 external_mini_cluster.cc:1428] Setting key cce394120d9ed21abcc8aacfc98978ea
I20260812 04:00:02.437772 13952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:34679 every 8 connection(s)
Aug 12 04:00:02 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507202, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:02.455212 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:02.438857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:51953 (local address 127.31.43.254:37343)
0812 04:00:02.438924 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:02.438926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:02.439238 (+ 312us) server_negotiation.cc:407] Connection header received
0812 04:00:02.440150 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:02.440151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:02.440169 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:02.440207 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:02.441250 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:02.442136 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:02.443856 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:02.444211 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:02.444264 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:02.448581 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:02.448583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:02.448583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:02.448592 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:02.450453 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:02.451782 (+ 1329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:02.451784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:02.451784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:02.451854 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:02.452194 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:02.452196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:02.452196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:02.452358 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:02.452428 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:02.454978 (+ 2550us) server_negotiation.cc:299] Negotiation successful
0812 04:00:02.455091 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 04:00:02.456127 13955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:02.438494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37343 (local address 127.31.43.194:51953)
0812 04:00:02.439097 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:02.439148 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:02.440011 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:02.440435 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:02.440440 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:02.440745 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:02.441120 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:02.441128 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:02.442267 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:02.442268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:02.443740 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:02.443743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:02.444381 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:02.444382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:02.444547 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:02.446422 (+ 1875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:02.446434 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:02.448444 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:02.450561 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:02.450569 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:02.450572 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:02.451003 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:02.451943 (+ 940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:02.451944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:02.451945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:02.452103 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:02.452469 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:02.452475 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:02.452658 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 04:00:02.455648 (+ 2990us) client_negotiation.cc:240] Negotiation successful
0812 04:00:02.455851 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":100,"threads_started":1}
I20260812 04:00:02.456909 13953 heartbeater.cc:347] Connected to a master server at 127.31.43.254:37343
I20260812 04:00:02.457018 13953 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:02.457239 13953 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
I20260812 04:00:02.457728 13555 ts_manager.cc:194] Registered new tserver with Master: 88109a58f1da4fe095ea1bdd424b2ba4 (127.31.43.194:34679)
I20260812 04:00:02.458344 13555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:51953
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 04:00:02.490535 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.43.212:42663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:37343
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:02.622046 13958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:02.622354 13958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:02.622398 13958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:02.623970 13958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:02.624050 13958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:02.624107 13958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:00:02.626075 13958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.43.212:42663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:37343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:02.626356 13958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:02.626611 13958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:02.632745 13966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:02.632704 13964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:02.632704 13963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:02.633637 13958 server_base.cc:1034] running on GCE node
I20260812 04:00:02.633885 13958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:02.634295 13958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:02.637590 13958 hybrid_clock.cc:648] HybridClock initialized: now 1786507202637571 us; error 28 us; skew 500 ppm
Aug 12 04:00:02 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507202, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:02.640829 13958 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 04:00:02.641556 13958 webserver.cc:460] Webserver started at http://127.31.43.195:40781/ using document root <none> and password file <none>
I20260812 04:00:02.641839 13958 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:02.641909 13958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:02.642055 13958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 04:00:02 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507202, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 04:00:02.659348 13958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "9c35750d83fd4f458d3e761eedbc072c"
format_stamp: "Formatted at 2026-08-12 04:00:02 on dist-test-slave-8tdl"
server_key: "1569fa7c0cba3c45c47d3a4576dadc04"
server_key_iv: "3e53b725fe06a7b5718ed9ff3336ec5d"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:02.659855 13958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "9c35750d83fd4f458d3e761eedbc072c"
format_stamp: "Formatted at 2026-08-12 04:00:02 on dist-test-slave-8tdl"
server_key: "1569fa7c0cba3c45c47d3a4576dadc04"
server_key_iv: "3e53b725fe06a7b5718ed9ff3336ec5d"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:02.661885 13958 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 04:00:02.697842 13973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:02.698112 13958 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:02.698228 13958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "9c35750d83fd4f458d3e761eedbc072c"
format_stamp: "Formatted at 2026-08-12 04:00:02 on dist-test-slave-8tdl"
server_key: "1569fa7c0cba3c45c47d3a4576dadc04"
server_key_iv: "3e53b725fe06a7b5718ed9ff3336ec5d"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:02.698356 13958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:02.710107 13958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:02.712492 13958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:02.712672 13958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:02.712978 13958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:02.713373 13958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:02.713429 13958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:02.713497 13958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:02.713527 13958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:02.721897 13958 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:38215
I20260812 04:00:02.722002 14103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:38215 every 8 connection(s)
I20260812 04:00:02.722333 13958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
Aug 12 04:00:02 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507202, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:02.729712 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 13958
I20260812 04:00:02.729816 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 04:00:02.730032 31919 external_mini_cluster.cc:1428] Setting key 3f43d0562690166fee57106f5cf0f62e
I20260812 04:00:02.733600 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:02.723189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:32881 (local address 127.31.43.254:37343)
0812 04:00:02.723263 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:02.723266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:02.723484 (+ 218us) server_negotiation.cc:407] Connection header received
0812 04:00:02.724077 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:02.724078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:02.724092 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:02.724156 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:02.724774 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:02.725374 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:02.726477 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:02.726766 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:02.726851 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:02.729675 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:02.729676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:02.729677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:02.729685 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:02.731779 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:02.732349 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:02.732350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:02.732351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:02.732421 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:02.732762 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:02.732764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:02.732765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:02.732934 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:02.733034 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:02.733391 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 04:00:02.733491 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 04:00:02.733773 14106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:02.723150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37343 (local address 127.31.43.195:32881)
0812 04:00:02.723351 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:02.723391 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:02.723960 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:02.724230 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:02.724234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:02.724465 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:02.724670 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:02.724675 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:02.725499 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:02.725500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:02.726356 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:02.726359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:02.726886 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:02.726888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:02.727039 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:02.727656 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:02.727666 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:02.729534 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:02.731882 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:02.731890 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:02.731893 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:02.732249 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:02.732512 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:02.732513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:02.732513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:02.732672 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:02.733032 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:02.733039 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:02.733210 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 04:00:02.733338 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 04:00:02.733512 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":117,"threads_started":1}
I20260812 04:00:02.734372 14104 heartbeater.cc:347] Connected to a master server at 127.31.43.254:37343
I20260812 04:00:02.734476 14104 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:02.734804 14104 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
I20260812 04:00:02.735328 13555 ts_manager.cc:194] Registered new tserver with Master: 9c35750d83fd4f458d3e761eedbc072c (127.31.43.195:38215)
I20260812 04:00:02.735893 13555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:32881
I20260812 04:00:02.742190 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 04:00:02.742995 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 13650
I20260812 04:00:02.761904 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 13804
I20260812 04:00:02.771142 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 13958
I20260812 04:00:02.780941 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 13521
I20260812 04:00:02.789587 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:37343
--webserver_interface=127.31.43.254
--webserver_port=46703
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:37343
--ranger_config_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.43.212:42663/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.43.212:42663/kms with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:02.873790 14107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:02.874063 14107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:02.874114 14107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:02.875351 14107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:00:02.875429 14107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:02.875455 14107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:02.875483 14107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:00:02.875514 14107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:00:02.877200 14107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.43.212:42663/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:37343
--ranger_config_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:37343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=46703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:02.877491 14107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:02.877739 14107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:02.883052 14114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:02.883059 14113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:02.883265 14107 server_base.cc:1034] running on GCE node
W20260812 04:00:02.883100 14116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:02.883582 14107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:02.883926 14107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:02.885077 14107 hybrid_clock.cc:648] HybridClock initialized: now 1786507202885056 us; error 42 us; skew 500 ppm
Aug 12 04:00:02 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507202, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:02.887898 14107 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:00:02.888417 14107 webserver.cc:460] Webserver started at http://127.31.43.254:46703/ using document root <none> and password file <none>
I20260812 04:00:02.888690 14107 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:02.888774 14107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 04:00:02 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507202, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 04:00:02.942554 14107 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:00:02.943673 14124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:02.943902 14107 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:02.944017 14107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "ab9c9ed359784d2182a02e4d6678c286"
format_stamp: "Formatted at 2026-08-12 04:00:01 on dist-test-slave-8tdl"
server_key: "dae951df93e9d0e420189db18f319369"
server_key_iv: "a2acfd60f14aa6728cc0c8ceddd9455b"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:02.944550 14107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:02.970855 14107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:02.972886 14107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:02.973093 14107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:02.978051 14107 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:37343
I20260812 04:00:02.978117 14184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:37343 every 8 connection(s)
I20260812 04:00:02.978524 14107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:00:02.979928 14185 sys_catalog.cc:263] Verifying existing consensus state
I20260812 04:00:02.980647 14185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286: Bootstrap starting.
I20260812 04:00:02.986899 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 14107
I20260812 04:00:02.987293 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.43.212:42663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:38315
--local_ip_for_outbound_sockets=127.31.43.193
--tserver_master_addrs=127.31.43.254:37343
--webserver_port=34111
--webserver_interface=127.31.43.193
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.43.212:42663/kms with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:00:02.990412 14185 log.cc:824] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:02.992477 14185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 04:00:02.992796 14185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286: Bootstrap complete.
I20260812 04:00:02.995505 14185 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9c9ed359784d2182a02e4d6678c286" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37343 } }
I20260812 04:00:02.995755 14185 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab9c9ed359784d2182a02e4d6678c286, State: Initialized, Role: FOLLOWER
I20260812 04:00:02.995934 14185 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9c9ed359784d2182a02e4d6678c286" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37343 } }
I20260812 04:00:02.996011 14185 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:00:02.996061 14185 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:00:02.996114 14185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 1 FOLLOWER]: Advancing to term 2
I20260812 04:00:02.997166 14185 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9c9ed359784d2182a02e4d6678c286" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37343 } }
I20260812 04:00:02.997263 14185 leader_election.cc:302] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab9c9ed359784d2182a02e4d6678c286; no voters:
I20260812 04:00:02.997524 14185 leader_election.cc:288] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 04:00:02.997563 14190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 04:00:02.997690 14190 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [term 2 LEADER]: Becoming Leader. State: Replica: ab9c9ed359784d2182a02e4d6678c286, State: Running, Role: LEADER
I20260812 04:00:02.997792 14190 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9c9ed359784d2182a02e4d6678c286" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37343 } }
I20260812 04:00:02.997821 14185 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:00:02.998090 14191 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab9c9ed359784d2182a02e4d6678c286. Latest consensus state: current_term: 2 leader_uuid: "ab9c9ed359784d2182a02e4d6678c286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9c9ed359784d2182a02e4d6678c286" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37343 } } }
I20260812 04:00:02.998325 14191 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:02.998253 14192 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ab9c9ed359784d2182a02e4d6678c286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9c9ed359784d2182a02e4d6678c286" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37343 } } }
I20260812 04:00:02.998481 14192 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286 [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:02.998728 14197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:00:02.999387 14197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:00:02.999598 14197 catalog_manager.cc:1259] Loaded cluster ID: efa30d77c34a4761b4d212f596484626
I20260812 04:00:02.999662 14197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:00:02.999826 14185 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 04:00:03.003652 14197 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:00:03.004320 14197 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286: Loaded TSK: 0
I20260812 04:00:03.004657 14197 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 04:00:03.078395 14188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:03.078806 14188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:03.078933 14188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:03.080857 14188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:03.081001 14188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:03.081101 14188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:00:03.083616 14188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.43.212:42663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:38315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=34111
--tserver_master_addrs=127.31.43.254:37343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:03.083904 14188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:03.084271 14188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:03.090153 14230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:03.090360 14228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:03.090499 14227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:03.090658 14188 server_base.cc:1034] running on GCE node
I20260812 04:00:03.090885 14188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:03.091254 14188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:03.092437 14188 hybrid_clock.cc:648] HybridClock initialized: now 1786507203092413 us; error 38 us; skew 500 ppm
Aug 12 04:00:03 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507203, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:03.095536 14188 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 04:00:03.096191 14188 webserver.cc:460] Webserver started at http://127.31.43.193:34111/ using document root <none> and password file <none>
I20260812 04:00:03.096570 14188 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:03.096720 14188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 04:00:03 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507203, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 04:00:03.118031 14188 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:03.123185 14239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:03.123399 14188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 04:00:03.123492 14188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "a2c43343d0e047138e6d6f4faf8616ea"
format_stamp: "Formatted at 2026-08-12 04:00:02 on dist-test-slave-8tdl"
server_key: "27e26230d74368a70543f988788568e6"
server_key_iv: "f33ec52ccf1a0ede246ccc623545c324"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:03.123919 14188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:03.133936 14188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:03.136586 14188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:03.136833 14188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:03.137157 14188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:03.137588 14188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:03.137661 14188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:03.137728 14188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:03.137773 14188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:03.146466 14188 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:38315
I20260812 04:00:03.147024 14188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 04:00:03.147763 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 14188
I20260812 04:00:03.148108 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.43.212:42663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:34679
--local_ip_for_outbound_sockets=127.31.43.194
--tserver_master_addrs=127.31.43.254:37343
--webserver_port=38433
--webserver_interface=127.31.43.194
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.43.212:42663/kms with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:00:03.147756 14365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:38315 every 8 connection(s)
Aug 12 04:00:03 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507203, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:03.167383 14368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:03.148985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37343 (local address 127.31.43.193:33635)
0812 04:00:03.154779 (+ 5794us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:03.154841 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:03.155578 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:03.156282 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:03.156287 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:03.156595 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:03.156975 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:03.156983 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.158505 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.158506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:03.159478 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:03.159481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.160185 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.160187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:03.160325 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:03.160957 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:03.160967 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:03.162441 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:03.165024 (+ 2583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:03.165032 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:03.165035 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:03.165392 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:03.165710 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:03.165711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:03.165712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:03.165863 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:03.166546 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:03.166554 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:03.166754 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 04:00:03.166878 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 04:00:03.167082 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5737,"thread_start_us":96,"threads_started":1}
I20260812 04:00:03.168587 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:03.149401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:33635 (local address 127.31.43.254:37343)
0812 04:00:03.155010 (+ 5609us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:03.155021 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:03.155076 (+ 55us) server_negotiation.cc:407] Connection header received
0812 04:00:03.155866 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:03.155874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:03.155939 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:03.156105 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:03.157159 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.158363 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.159596 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.160050 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.160197 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:03.162610 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:03.162615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:03.162615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:03.162628 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:03.164906 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:03.165501 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:03.165504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:03.165505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:03.165631 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:03.165941 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:03.165942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:03.165942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:03.166417 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:03.166508 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:03.168131 (+ 1623us) server_negotiation.cc:299] Negotiation successful
0812 04:00:03.168294 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5541,"thread_start_us":106,"threads_started":1}
I20260812 04:00:03.169669 14366 heartbeater.cc:347] Connected to a master server at 127.31.43.254:37343
I20260812 04:00:03.169801 14366 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:03.170012 14366 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
W20260812 04:00:03.171047 14366 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:37343 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 04:00:03.171144 14366 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:03.171221 14366 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
I20260812 04:00:03.171663 14366 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:03.171748 14366 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
W20260812 04:00:03.172099 14366 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 04:00:03.301726 14370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:03.302067 14370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:03.302119 14370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:03.304183 14370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:03.304258 14370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:03.304309 14370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:00:03.306859 14370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.43.212:42663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:34679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=38433
--tserver_master_addrs=127.31.43.254:37343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:03.307173 14370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:03.307550 14370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:03.313766 14378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:03.314121 14377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:03.314502 14370 server_base.cc:1034] running on GCE node
W20260812 04:00:03.315408 14380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:03.315733 14370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:03.316088 14370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:03.317327 14370 hybrid_clock.cc:648] HybridClock initialized: now 1786507203317318 us; error 38 us; skew 500 ppm
Aug 12 04:00:03 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507203, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:03.320502 14370 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 04:00:03.321048 14370 webserver.cc:460] Webserver started at http://127.31.43.194:38433/ using document root <none> and password file <none>
I20260812 04:00:03.321346 14370 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:03.321452 14370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 04:00:03 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507203, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 04:00:03.346186 14370 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:03.347390 14388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:03.347633 14370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:03.347723 14370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "88109a58f1da4fe095ea1bdd424b2ba4"
format_stamp: "Formatted at 2026-08-12 04:00:02 on dist-test-slave-8tdl"
server_key: "e6c9be3827b4f83096e280e5e3a352c0"
server_key_iv: "6054ada7714a33a2b7a57d9a28a2a486"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:03.348238 14370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:03.376789 14370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:03.381482 14370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:03.381673 14370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:03.381906 14370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:03.382373 14370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:03.382429 14370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:03.382464 14370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:03.382485 14370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:03.390288 14370 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:34679
I20260812 04:00:03.390832 14370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:00:03.395541 14518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:34679 every 8 connection(s)
I20260812 04:00:03.396162 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 14370
I20260812 04:00:03.396510 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.43.212:42663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:38215
--local_ip_for_outbound_sockets=127.31.43.195
--tserver_master_addrs=127.31.43.254:37343
--webserver_port=40781
--webserver_interface=127.31.43.195
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.43.212:42663/kms with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 04:00:03 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507203, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:03.410492 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:03.398617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:60889 (local address 127.31.43.254:37343)
0812 04:00:03.398688 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:03.398690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:03.398702 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:00:03.398735 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:03.398736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:03.398752 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:03.398832 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:03.399910 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.400752 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.402557 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.402986 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.403043 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:03.406567 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:03.406570 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:03.406570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:03.406580 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:03.408520 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:03.409131 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:03.409132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:03.409133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:03.409202 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:03.409533 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:03.409535 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:03.409535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:03.409720 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:03.409774 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:03.410213 (+ 439us) server_negotiation.cc:299] Negotiation successful
0812 04:00:03.410328 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 04:00:03.411494 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:03.396301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37343 (local address 127.31.43.194:60889)
0812 04:00:03.396480 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:03.396530 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:03.397053 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:03.398954 (+ 1901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:03.398960 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:03.399385 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:03.399761 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:03.399768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.400887 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.400888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:03.402433 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:03.402437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.403185 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.403186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:03.403381 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:03.404356 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:03.404369 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:03.406415 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:03.408640 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:03.408647 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:03.408650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:03.409023 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:03.409283 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:03.409284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:03.409284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:03.409444 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:03.409828 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:03.409835 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:03.410028 (+ 193us) client_negotiation.cc:769] Sending connection context
0812 04:00:03.410975 (+ 947us) client_negotiation.cc:240] Negotiation successful
0812 04:00:03.411178 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260812 04:00:03.412273 14519 heartbeater.cc:347] Connected to a master server at 127.31.43.254:37343
I20260812 04:00:03.412403 14519 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:03.412639 14519 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
W20260812 04:00:03.413504 14519 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:37343 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 04:00:03.413595 14519 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:03.413678 14519 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
I20260812 04:00:03.414065 14519 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:03.414152 14519 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
W20260812 04:00:03.414610 14519 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 04:00:03.500881 14522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:03.501155 14522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:03.501200 14522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:03.502490 14522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:03.502578 14522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:03.502628 14522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:00:03.504976 14522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.43.212:42663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:38215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=40781
--tserver_master_addrs=127.31.43.254:37343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:03.505388 14522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:03.505663 14522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:03.511699 14528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:03.513041 14527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:03.513340 14530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:03.514796 14522 server_base.cc:1034] running on GCE node
I20260812 04:00:03.515228 14522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:03.515578 14522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:03.518877 14522 hybrid_clock.cc:648] HybridClock initialized: now 1786507203518844 us; error 41 us; skew 500 ppm
Aug 12 04:00:03 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507203, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:03.522167 14522 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 04:00:03.522827 14522 webserver.cc:460] Webserver started at http://127.31.43.195:40781/ using document root <none> and password file <none>
I20260812 04:00:03.523186 14522 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:03.523353 14522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 04:00:03 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507203, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 04:00:03.548276 14522 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260812 04:00:03.549177 14539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:03.549378 14522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:03.549463 14522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "9c35750d83fd4f458d3e761eedbc072c"
format_stamp: "Formatted at 2026-08-12 04:00:02 on dist-test-slave-8tdl"
server_key: "1569fa7c0cba3c45c47d3a4576dadc04"
server_key_iv: "3e53b725fe06a7b5718ed9ff3336ec5d"
server_key_version: "kuduclusterkey@0"
I20260812 04:00:03.550889 14522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:03.598280 14522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:03.600834 14522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:03.601033 14522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:03.601339 14522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:03.601748 14522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:03.601861 14522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:03.601943 14522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:03.602027 14522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:03.609858 14522 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:38215
I20260812 04:00:03.610075 14664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:38215 every 8 connection(s)
I20260812 04:00:03.610462 14522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 04:00:03.615413 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 14522
Aug 12 04:00:03 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507203, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:03.625320 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:03.611632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:37343 (local address 127.31.43.195:37667)
0812 04:00:03.611911 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:03.611966 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:03.612604 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:03.612904 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:03.612909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:03.613196 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:03.613479 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:03.613485 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.614556 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.614557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:03.616444 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:03.616447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.617072 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.617073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:03.617241 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:03.617872 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:03.617883 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:03.620379 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:03.623348 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:03.623359 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:03.623368 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:03.623706 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:03.623966 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:03.623967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:03.623967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:03.624124 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:03.624478 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:03.624486 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:03.624677 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 04:00:03.624840 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 04:00:03.625051 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":84,"threads_started":1}
I20260812 04:00:03.625384 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:03.611642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:37667 (local address 127.31.43.254:37343)
0812 04:00:03.611693 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:03.611695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:03.612080 (+ 385us) server_negotiation.cc:407] Connection header received
0812 04:00:03.612726 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:03.612728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:03.612742 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:03.612821 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:03.613612 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.614425 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.616564 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.616875 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.616945 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:03.621066 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:03.621068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:03.621068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:03.621077 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:03.623267 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:03.623805 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:03.623807 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:03.623807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:03.623878 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:03.624212 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:03.624213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:03.624213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:03.624362 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:03.624412 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:03.624890 (+ 478us) server_negotiation.cc:299] Negotiation successful
0812 04:00:03.625027 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 04:00:03.626174 14665 heartbeater.cc:347] Connected to a master server at 127.31.43.254:37343
I20260812 04:00:03.626307 14665 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:03.626510 14665 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
W20260812 04:00:03.627332 14665 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:37343 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 04:00:03.627417 14665 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:03.627507 14665 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
I20260812 04:00:03.627811 14665 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:03.627892 14665 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
I20260812 04:00:03.628178 14669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:03.619445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50704 (local address 127.31.43.254:37343)
0812 04:00:03.619596 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:03.619599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:03.619608 (+ 9us) server_negotiation.cc:407] Connection header received
0812 04:00:03.619634 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:03.619635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:03.619647 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:03.619720 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:03.620871 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.621632 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.622819 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:03.623166 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:03.623250 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:03.624799 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:03.624800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:03.624801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:03.624809 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:03.626741 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:03.627198 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:03.627199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:03.627199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:03.627271 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:03.627509 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:03.627510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:03.627511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:03.627651 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:03.627833 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:03.627934 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 04:00:03.628028 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
W20260812 04:00:03.628233 14665 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 04:00:04.172330 14366 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:04.172505 14366 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
I20260812 04:00:04.172959 14141 ts_manager.cc:194] Registered new tserver with Master: a2c43343d0e047138e6d6f4faf8616ea (127.31.43.193:38315)
I20260812 04:00:04.174132 14141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:33635
Aug 12 04:00:04 dist-test-slave-8tdl krb5kdc[12999](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786507204, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:04.414775 14519 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:04.414963 14519 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
I20260812 04:00:04.415297 14141 ts_manager.cc:194] Registered new tserver with Master: 88109a58f1da4fe095ea1bdd424b2ba4 (127.31.43.194:34679)
I20260812 04:00:04.415840 14141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:60889
I20260812 04:00:04.628425 14665 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:04.628602 14665 heartbeater.cc:511] Master 127.31.43.254:37343 requested a full tablet report, sending...
I20260812 04:00:04.628989 14141 ts_manager.cc:194] Registered new tserver with Master: 9c35750d83fd4f458d3e761eedbc072c (127.31.43.195:38215)
I20260812 04:00:04.629498 14141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:37667
I20260812 04:00:04.638345 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 04:00:04 dist-test-slave-8tdl krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507204, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for HTTP/127.31.43.212@KRBTEST.COM
I20260812 04:00:04.650013 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.639699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50770 (local address 127.31.43.254:37343)
0812 04:00:04.639980 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:04.639983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:04.640166 (+ 183us) server_negotiation.cc:407] Connection header received
0812 04:00:04.640194 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:04.640195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:04.640210 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:04.640276 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:04.641070 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.642310 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.643648 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.643938 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.644011 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:04.645673 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:04.645675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:04.645675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:04.645683 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:04.647589 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:04.648629 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:04.648630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:04.648631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:04.648701 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:04.648989 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:04.648990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:04.648990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:04.649136 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:04.649232 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:04.649759 (+ 527us) server_negotiation.cc:299] Negotiation successful
0812 04:00:04.649849 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":69,"threads_started":1}
I20260812 04:00:04.652405 14141 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 04:00:04.655191 14141 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 04:00:04.671804 14697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.662623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:34679 (local address 127.0.0.1:36140)
0812 04:00:04.666964 (+ 4341us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:04.667028 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:04.667111 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:04.667625 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:04.667628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:04.667631 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:04.667729 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:04.667732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.669067 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.669068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:04.670423 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:04.670427 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.671543 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.671544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:04.671620 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:04.671632 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 04:00:04.671683 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 04:00:04.671724 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4300,"thread_start_us":104,"threads_started":1}
I20260812 04:00:04.673416 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.662915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36140 (local address 127.31.43.194:34679)
0812 04:00:04.667272 (+ 4357us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:04.667277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:04.667288 (+ 11us) server_negotiation.cc:407] Connection header received
0812 04:00:04.667327 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:04.667330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:04.667348 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:04.667424 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:04.667830 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.668960 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.670544 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.671391 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.671481 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:04.673258 (+ 1777us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:04.673284 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 04:00:04.673330 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4317,"thread_start_us":61,"threads_started":1}
I20260812 04:00:04.674396 14444 tablet_service.cc:1467] Processing CreateTablet for tablet e083925d0633432091925182c62c9514 (DEFAULT_TABLE table=test-table [id=c1c6b9a991644585b39f2e2535518cb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:04.674916 14444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e083925d0633432091925182c62c9514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:04.675359 14697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.662623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:38315 (local address 127.0.0.1:60400)
0812 04:00:04.672041 (+ 9418us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:04.672052 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:04.672089 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:04.672316 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:04.672317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:04.672319 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:04.672357 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:04.672358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.673182 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.673183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:04.674240 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:04.674242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.675121 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.675122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:04.675179 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:04.675182 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:00:04.675244 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 04:00:04.675283 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9405,"mutex_wait_us":2,"thread_start_us":138,"threads_started":1}
I20260812 04:00:04.675372 14700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.663108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60400 (local address 127.31.43.193:38315)
0812 04:00:04.663250 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:04.663255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:04.672154 (+ 8899us) server_negotiation.cc:407] Connection header received
0812 04:00:04.672201 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:04.672204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:04.672229 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:04.672323 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:04.672418 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.673083 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.674339 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.675048 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.675124 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:04.675184 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:04.675258 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 04:00:04.675297 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":86,"threads_started":1}
I20260812 04:00:04.676501 14293 tablet_service.cc:1467] Processing CreateTablet for tablet e083925d0633432091925182c62c9514 (DEFAULT_TABLE table=test-table [id=c1c6b9a991644585b39f2e2535518cb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:04.676879 14293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e083925d0633432091925182c62c9514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:04.678704 14701 tablet_bootstrap.cc:492] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: Bootstrap starting.
I20260812 04:00:04.679175 14697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.662888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:38215 (local address 127.0.0.1:39508)
0812 04:00:04.675818 (+ 12930us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:04.675832 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:04.675933 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:04.676199 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:04.676200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:04.676201 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:04.676225 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:04.676226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.677179 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.677180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:04.678030 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:04.678033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.678906 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.678907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:04.678981 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:04.678985 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:00:04.679044 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 04:00:04.679092 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12914,"thread_start_us":92,"threads_started":1}
I20260812 04:00:04.679665 14701 tablet_bootstrap.cc:654] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:04.679804 14699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.663000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39508 (local address 127.31.43.195:38215)
0812 04:00:04.666862 (+ 3862us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:04.666868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:04.675895 (+ 9027us) server_negotiation.cc:407] Connection header received
0812 04:00:04.676029 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:04.676033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:04.676052 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:04.676109 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:04.676296 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.677102 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.678122 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.678737 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.678808 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:04.678869 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:04.679675 (+ 806us) server_negotiation.cc:299] Negotiation successful
0812 04:00:04.679727 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3813,"thread_start_us":55,"threads_started":1}
I20260812 04:00:04.680225 14701 log.cc:824] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:04.680461 14588 tablet_service.cc:1467] Processing CreateTablet for tablet e083925d0633432091925182c62c9514 (DEFAULT_TABLE table=test-table [id=c1c6b9a991644585b39f2e2535518cb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:04.680800 14588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e083925d0633432091925182c62c9514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:04.681144 14701 tablet_bootstrap.cc:492] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: No bootstrap required, opened a new log
I20260812 04:00:04.681209 14701 ts_tablet_manager.cc:1397] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 04:00:04.683924 14701 raft_consensus.cc:348] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c43343d0e047138e6d6f4faf8616ea" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38315 } } peers { permanent_uuid: "9c35750d83fd4f458d3e761eedbc072c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38215 } } peers { permanent_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 34679 } }
I20260812 04:00:04.684098 14701 raft_consensus.cc:374] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:04.684137 14701 raft_consensus.cc:729] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88109a58f1da4fe095ea1bdd424b2ba4, State: Initialized, Role: FOLLOWER
I20260812 04:00:04.684434 14701 consensus_queue.cc:260] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c43343d0e047138e6d6f4faf8616ea" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38315 } } peers { permanent_uuid: "9c35750d83fd4f458d3e761eedbc072c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38215 } } peers { permanent_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 34679 } }
I20260812 04:00:04.684767 14701 ts_tablet_manager.cc:1428] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 04:00:04.684916 14519 heartbeater.cc:503] Master 127.31.43.254:37343 was elected leader, sending a full tablet report...
I20260812 04:00:04.685019 14704 tablet_bootstrap.cc:492] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c: Bootstrap starting.
I20260812 04:00:04.685954 14704 tablet_bootstrap.cc:654] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:04.686371 14704 log.cc:824] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:04.687338 14704 tablet_bootstrap.cc:492] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c: No bootstrap required, opened a new log
I20260812 04:00:04.687407 14704 ts_tablet_manager.cc:1397] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 04:00:04.690083 14704 raft_consensus.cc:348] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c43343d0e047138e6d6f4faf8616ea" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38315 } } peers { permanent_uuid: "9c35750d83fd4f458d3e761eedbc072c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38215 } } peers { permanent_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 34679 } }
I20260812 04:00:04.690263 14704 raft_consensus.cc:374] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:04.690303 14704 raft_consensus.cc:729] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c35750d83fd4f458d3e761eedbc072c, State: Initialized, Role: FOLLOWER
I20260812 04:00:04.690444 14704 consensus_queue.cc:260] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c43343d0e047138e6d6f4faf8616ea" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38315 } } peers { permanent_uuid: "9c35750d83fd4f458d3e761eedbc072c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38215 } } peers { permanent_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 34679 } }
I20260812 04:00:04.690788 14704 ts_tablet_manager.cc:1428] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 04:00:04.690914 14665 heartbeater.cc:503] Master 127.31.43.254:37343 was elected leader, sending a full tablet report...
I20260812 04:00:04.691336 14707 tablet_bootstrap.cc:492] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea: Bootstrap starting.
I20260812 04:00:04.692139 14707 tablet_bootstrap.cc:654] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:04.692500 14707 log.cc:824] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:04.693367 14707 tablet_bootstrap.cc:492] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea: No bootstrap required, opened a new log
I20260812 04:00:04.693442 14707 ts_tablet_manager.cc:1397] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 04:00:04.695719 14707 raft_consensus.cc:348] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c43343d0e047138e6d6f4faf8616ea" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38315 } } peers { permanent_uuid: "9c35750d83fd4f458d3e761eedbc072c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38215 } } peers { permanent_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 34679 } }
I20260812 04:00:04.695883 14707 raft_consensus.cc:374] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:04.695914 14707 raft_consensus.cc:729] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c43343d0e047138e6d6f4faf8616ea, State: Initialized, Role: FOLLOWER
I20260812 04:00:04.696492 14707 consensus_queue.cc:260] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c43343d0e047138e6d6f4faf8616ea" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38315 } } peers { permanent_uuid: "9c35750d83fd4f458d3e761eedbc072c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38215 } } peers { permanent_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 34679 } }
I20260812 04:00:04.696771 14707 ts_tablet_manager.cc:1428] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 04:00:04.697091 14366 heartbeater.cc:503] Master 127.31.43.254:37343 was elected leader, sending a full tablet report...
I20260812 04:00:04.806059 14708 raft_consensus.cc:482] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:00:04.806241 14708 raft_consensus.cc:504] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c43343d0e047138e6d6f4faf8616ea" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38315 } } peers { permanent_uuid: "9c35750d83fd4f458d3e761eedbc072c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38215 } } peers { permanent_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 34679 } }
I20260812 04:00:04.806761 14708 leader_election.cc:288] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2c43343d0e047138e6d6f4faf8616ea (127.31.43.193:38315), 88109a58f1da4fe095ea1bdd424b2ba4 (127.31.43.194:34679)
I20260812 04:00:04.811193 14700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.807207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:51447 (local address 127.31.43.193:38315)
0812 04:00:04.807253 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:04.807256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:04.807451 (+ 195us) server_negotiation.cc:407] Connection header received
0812 04:00:04.807570 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:04.807571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:04.807580 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:04.807612 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:04.807882 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.808619 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.810162 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.810767 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.810819 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:04.810829 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:04.811076 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 04:00:04.811116 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 04:00:04.811579 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.806952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:38315 (local address 127.31.43.195:51447)
0812 04:00:04.807368 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:04.807380 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:04.807487 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:04.807694 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:04.807695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:04.807697 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:04.807789 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:04.807791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.808727 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.808728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:04.810054 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:04.810057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.810919 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.810920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:04.810992 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:04.810995 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:00:04.811403 (+ 408us) client_negotiation.cc:240] Negotiation successful
0812 04:00:04.811465 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":24,"thread_start_us":124,"threads_started":1}
I20260812 04:00:04.811609 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.807257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:48269 (local address 127.31.43.194:34679)
0812 04:00:04.807302 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:04.807304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:04.807467 (+ 163us) server_negotiation.cc:407] Connection header received
0812 04:00:04.807573 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:04.807574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:04.807583 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:04.807613 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:04.807864 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.808590 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.810164 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.810768 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.810819 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:04.810828 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:04.811484 (+ 656us) server_negotiation.cc:299] Negotiation successful
0812 04:00:04.811536 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 04:00:04.812104 14315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e083925d0633432091925182c62c9514" candidate_uuid: "9c35750d83fd4f458d3e761eedbc072c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c43343d0e047138e6d6f4faf8616ea" is_pre_election: true
I20260812 04:00:04.812292 14315 raft_consensus.cc:2454] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c35750d83fd4f458d3e761eedbc072c in term 0.
I20260812 04:00:04.812444 14712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.807007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:34679 (local address 127.31.43.195:48269)
0812 04:00:04.807387 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:04.807398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:04.807497 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:04.807687 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:04.807687 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:04.807689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:04.807780 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:04.807783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.808696 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.808696 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:04.810053 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:04.810056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.811172 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.811173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:04.811260 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:04.811263 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:00:04.811837 (+ 574us) client_negotiation.cc:240] Negotiation successful
0812 04:00:04.811881 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":47,"thread_start_us":116,"threads_started":1}
I20260812 04:00:04.812572 14544 leader_election.cc:302] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c35750d83fd4f458d3e761eedbc072c, a2c43343d0e047138e6d6f4faf8616ea; no voters:
I20260812 04:00:04.812721 14708 raft_consensus.cc:2781] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 04:00:04.812770 14708 raft_consensus.cc:482] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 04:00:04.812793 14708 raft_consensus.cc:3037] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:04.812963 14467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e083925d0633432091925182c62c9514" candidate_uuid: "9c35750d83fd4f458d3e761eedbc072c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" is_pre_election: true
I20260812 04:00:04.813114 14467 raft_consensus.cc:2454] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c35750d83fd4f458d3e761eedbc072c in term 0.
I20260812 04:00:04.813827 14708 raft_consensus.cc:504] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c43343d0e047138e6d6f4faf8616ea" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38315 } } peers { permanent_uuid: "9c35750d83fd4f458d3e761eedbc072c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38215 } } peers { permanent_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 34679 } }
I20260812 04:00:04.813980 14708 leader_election.cc:288] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [CANDIDATE]: Term 1 election: Requested vote from peers a2c43343d0e047138e6d6f4faf8616ea (127.31.43.193:38315), 88109a58f1da4fe095ea1bdd424b2ba4 (127.31.43.194:34679)
I20260812 04:00:04.814143 14467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e083925d0633432091925182c62c9514" candidate_uuid: "9c35750d83fd4f458d3e761eedbc072c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88109a58f1da4fe095ea1bdd424b2ba4"
I20260812 04:00:04.814221 14467 raft_consensus.cc:3037] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:04.814215 14315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e083925d0633432091925182c62c9514" candidate_uuid: "9c35750d83fd4f458d3e761eedbc072c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c43343d0e047138e6d6f4faf8616ea"
I20260812 04:00:04.814297 14315 raft_consensus.cc:3037] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:04.815300 14467 raft_consensus.cc:2454] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c35750d83fd4f458d3e761eedbc072c in term 1.
I20260812 04:00:04.815300 14315 raft_consensus.cc:2454] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c35750d83fd4f458d3e761eedbc072c in term 1.
I20260812 04:00:04.815506 14544 leader_election.cc:302] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c35750d83fd4f458d3e761eedbc072c, a2c43343d0e047138e6d6f4faf8616ea; no voters:
I20260812 04:00:04.815585 14708 raft_consensus.cc:2781] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:00:04.815758 14708 raft_consensus.cc:686] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 1 LEADER]: Becoming Leader. State: Replica: 9c35750d83fd4f458d3e761eedbc072c, State: Running, Role: LEADER
I20260812 04:00:04.815855 14708 consensus_queue.cc:237] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c43343d0e047138e6d6f4faf8616ea" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38315 } } peers { permanent_uuid: "9c35750d83fd4f458d3e761eedbc072c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38215 } } peers { permanent_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 34679 } }
I20260812 04:00:04.816779 14141 catalog_manager.cc:5495] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c35750d83fd4f458d3e761eedbc072c (127.31.43.195). New cstate: current_term: 1 leader_uuid: "9c35750d83fd4f458d3e761eedbc072c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c43343d0e047138e6d6f4faf8616ea" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c35750d83fd4f458d3e761eedbc072c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 38215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 34679 } health_report { overall_health: UNKNOWN } } }
I20260812 04:00:04.857684 14699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.854315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39514 (local address 127.31.43.195:38215)
0812 04:00:04.854361 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:04.854363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:04.854387 (+ 24us) server_negotiation.cc:407] Connection header received
0812 04:00:04.854439 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:04.854440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:04.854461 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:04.854532 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:00:04.854812 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.855623 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.856757 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:04.857034 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:04.857104 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:04.857303 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:00:04.857447 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:00:04.857528 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:04.857582 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 04:00:04.857606 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 04:00:04.861891 14666 tablet.cc:2369] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:00:04.862605 14467 raft_consensus.cc:1261] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4 [term 1 FOLLOWER]: Refusing update from remote peer 9c35750d83fd4f458d3e761eedbc072c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:00:04.862617 14315 raft_consensus.cc:1261] T e083925d0633432091925182c62c9514 P a2c43343d0e047138e6d6f4faf8616ea [term 1 FOLLOWER]: Refusing update from remote peer 9c35750d83fd4f458d3e761eedbc072c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:00:04.863010 14708 consensus_queue.cc:1035] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2c43343d0e047138e6d6f4faf8616ea" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 38315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:00:04.863212 14708 consensus_queue.cc:1035] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 34679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:00:04.866655 14716 mvcc.cc:205] Tried to move back new op lower bound from 7317533511105163264 to 7317533510926450688. Current Snapshot: MvccSnapshot[applied={T|T < 7317533511105163264}]
I20260812 04:00:04.870766 14140 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:00:04.870896 14140 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:00:04.871189 14718 mvcc.cc:205] Tried to move back new op lower bound from 7317533511105163264 to 7317533510926450688. Current Snapshot: MvccSnapshot[applied={T|T < 7317533511105163264}]
I20260812 04:00:04.871702 14140 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab9c9ed359784d2182a02e4d6678c286: Sending DeleteTablet for 3 replicas of tablet e083925d0633432091925182c62c9514
I20260812 04:00:04.872107 14444 tablet_service.cc:1514] Processing DeleteTablet for tablet e083925d0633432091925182c62c9514 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:04 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:36140
I20260812 04:00:04.872226 14293 tablet_service.cc:1514] Processing DeleteTablet for tablet e083925d0633432091925182c62c9514 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:04 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:60400
I20260812 04:00:04.872481 14724 tablet_replica.cc:330] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: stopping tablet replica
I20260812 04:00:04.872460 14588 tablet_service.cc:1514] Processing DeleteTablet for tablet e083925d0633432091925182c62c9514 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:04 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:39508
I20260812 04:00:04.872673 14724 raft_consensus.cc:2229] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:00:04.872823 14724 pending_rounds.cc:62] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: Trying to abort 2 pending ops.
I20260812 04:00:04.872970 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 14188
I20260812 04:00:04.872907 14724 pending_rounds.cc:68] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317533511105163264 op_type: WRITE_OP write_request { tablet_id: "e083925d0633432091925182c62c9514" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\360\341\357\323\006\"8\n\ntest-admin\022*\n c1c6b9a991644585b39f2e2535518cb2\020\001\030\001 \001(\001" signature: "<redacted>""Q\247|#\006\301\002\202\006\232\357)F\326\0343\373B]`Y@\031\214$`\314\'\225\222\305\216\330\216i\002+\344\332\354\314\335l\333\264\2602bJy6B\007\366{\311\220\014\240N-4\202\362" signing_key_seq_num: 0 } } request_id { client_id: "656341cd4f10479486a3cb7c1998df0b" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260812 04:00:04.873134 14724 pending_rounds.cc:68] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317533510926450688 op_type: NO_OP noop_request { }
I20260812 04:00:04.873174 14724 raft_consensus.cc:2866] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260812 04:00:04.873202 14724 raft_consensus.cc:2258] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 04:00:04.878233 14719 consensus_queue.cc:1159] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "88109a58f1da4fe095ea1bdd424b2ba4" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260812 04:00:04.878849 14724 ts_tablet_manager.cc:1905] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 04:00:04.879534 14540 consensus_peers.cc:487] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c -> Peer 88109a58f1da4fe095ea1bdd424b2ba4 (127.31.43.194:34679): Couldn't send request to peer 88109a58f1da4fe095ea1bdd424b2ba4. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260812 04:00:04.880935 14724 ts_tablet_manager.cc:1918] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:00:04.881083 14724 log.cc:1196] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/e083925d0633432091925182c62c9514
I20260812 04:00:04.881481 14724 ts_tablet_manager.cc:1939] T e083925d0633432091925182c62c9514 P 88109a58f1da4fe095ea1bdd424b2ba4: Deleting consensus metadata
I20260812 04:00:04.882066 14125 catalog_manager.cc:4841] TS 88109a58f1da4fe095ea1bdd424b2ba4 (127.31.43.194:34679): tablet e083925d0633432091925182c62c9514 (table test-table [id=c1c6b9a991644585b39f2e2535518cb2]) successfully deleted
I20260812 04:00:04.885800 14728 tablet_replica.cc:330] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c: stopping tablet replica
I20260812 04:00:04.886126 14728 raft_consensus.cc:2229] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 1 LEADER]: Raft consensus shutting down.
I20260812 04:00:04.886371 14728 raft_consensus.cc:2258] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:00:04.887234 14728 ts_tablet_manager.cc:1905] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:00:04.889302 14728 ts_tablet_manager.cc:1918] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:00:04.891649 14728 log.cc:1196] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/e083925d0633432091925182c62c9514
I20260812 04:00:04.892019 14728 ts_tablet_manager.cc:1939] T e083925d0633432091925182c62c9514 P 9c35750d83fd4f458d3e761eedbc072c: Deleting consensus metadata
I20260812 04:00:04.892612 14128 catalog_manager.cc:4841] TS 9c35750d83fd4f458d3e761eedbc072c (127.31.43.195:38215): tablet e083925d0633432091925182c62c9514 (table test-table [id=c1c6b9a991644585b39f2e2535518cb2]) successfully deleted
W20260812 04:00:04.893157 14128 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:38315 (error 108)
I20260812 04:00:04.894194 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:04.894042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:38315 (local address 127.0.0.1:60412)
0812 04:00:04.894097 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:04.894152 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:38315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 04:00:04.894204 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 14370
W20260812 04:00:04.894833 14128 catalog_manager.cc:4584] TS a2c43343d0e047138e6d6f4faf8616ea (127.31.43.193:38315): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e083925d0633432091925182c62c9514: Network error: Client connection negotiation failed: client connection to 127.31.43.193:38315: connect: Connection refused (error 111)
I20260812 04:00:04.904253 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 14522
I20260812 04:00:04.915668 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 14107
I20260812 04:00:04.924763 31919 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 04:00:05.270059 31919 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 04:00:05.270174 31919 mini_ranger.cc:66] Stopping Ranger...
I20260812 04:00:05.625501 31919 mini_ranger.cc:68] Stopped Ranger
2026-08-12 04:00:05.625 UTC [13032] LOG: received smart shutdown request
2026-08-12 04:00:05.631 UTC [13032] LOG: background worker "logical replication launcher" (PID 13042) exited with exit code 1
2026-08-12 04:00:05.631 UTC [13036] LOG: shutting down
2026-08-12 04:00:05.843 UTC [13032] LOG: database system is shut down
2026-08-12T04:00:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51362 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:00:06 dist-test-slave-8tdl krb5kdc[14739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:00:06 dist-test-slave-8tdl krb5kdc[14739](info): set up 2 sockets
Aug 12 04:00:06 dist-test-slave-8tdl krb5kdc[14739](info): commencing operation
krb5kdc: starting...
W20260812 04:00:07.125820 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:00:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:00:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:00:07.258132 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:43117
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:38165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:43117,127.31.43.253:35567,127.31.43.252:36551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:07.345678 14755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:07.345892 14755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:07.345916 14755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:07.347115 14755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:00:07.347160 14755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:07.347175 14755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:07.347190 14755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:00:07.347203 14755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:00:07.348744 14755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:38165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:43117,127.31.43.253:35567,127.31.43.252:36551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:43117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:07.348951 14755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:07.349143 14755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:07.354360 14764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:07.354439 14755 server_base.cc:1034] running on GCE node
W20260812 04:00:07.354386 14761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:07.354377 14762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:07.355055 14755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:07.355412 14755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:07.356591 14755 hybrid_clock.cc:648] HybridClock initialized: now 1786507207356565 us; error 40 us; skew 500 ppm
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:07.359349 14755 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:00:07.359884 14755 webserver.cc:460] Webserver started at http://127.31.43.254:41303/ using document root <none> and password file <none>
I20260812 04:00:07.360163 14755 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:07.360240 14755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:07.360399 14755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:00:07.361577 14755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "8d10f64a9854451497053ac6808752de"
format_stamp: "Formatted at 2026-08-12 04:00:07 on dist-test-slave-8tdl"
server_key: "efc2072bfd48ccfe3c894eb144f35294"
server_key_iv: "21c532ea05de0cfac67ca210f194c2ec"
server_key_version: "encryptionkey@0"
I20260812 04:00:07.361971 14755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "8d10f64a9854451497053ac6808752de"
format_stamp: "Formatted at 2026-08-12 04:00:07 on dist-test-slave-8tdl"
server_key: "efc2072bfd48ccfe3c894eb144f35294"
server_key_iv: "21c532ea05de0cfac67ca210f194c2ec"
server_key_version: "encryptionkey@0"
I20260812 04:00:07.363479 14755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:00:07.364428 14771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:07.364630 14755 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:07.364744 14755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "8d10f64a9854451497053ac6808752de"
format_stamp: "Formatted at 2026-08-12 04:00:07 on dist-test-slave-8tdl"
server_key: "efc2072bfd48ccfe3c894eb144f35294"
server_key_iv: "21c532ea05de0cfac67ca210f194c2ec"
server_key_version: "encryptionkey@0"
I20260812 04:00:07.364821 14755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:07.392400 14755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:07.394593 14755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:07.394797 14755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:07.399590 14755 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:43117
I20260812 04:00:07.399626 14830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:43117 every 8 connection(s)
I20260812 04:00:07.400007 14755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:00:07.400929 14831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:07.402559 14831 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:00:07.404582 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 14755
I20260812 04:00:07.404717 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:00:07.404948 31919 external_mini_cluster.cc:1428] Setting key c5e82d01d762e6d416a3649b6ed978be
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:07.414151 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.403338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:43117 (local address 127.0.0.1:47030)
0812 04:00:07.403530 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.403577 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:07.404072 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:07.404361 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:07.404363 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:07.404619 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:07.404863 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.404874 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.406375 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.406376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.407446 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.407450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.407999 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.408000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.408124 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.408738 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:07.408747 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:07.410318 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:07.412544 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.412551 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.412553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.412753 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.413054 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.413054 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.413055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.413158 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.413646 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:07.413652 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:07.413710 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 04:00:07.413803 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 04:00:07.413951 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260812 04:00:07.414186 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.403380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47030 (local address 127.31.43.254:43117)
0812 04:00:07.403692 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.403696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.403727 (+ 31us) server_negotiation.cc:407] Connection header received
0812 04:00:07.404224 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.404226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.404245 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.404324 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.404994 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.406269 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.407537 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.407856 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.408007 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.410416 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.410427 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.410427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.410437 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.412457 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.412844 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.412848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.412849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.412975 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.413233 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.413234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.413234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.413566 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.413669 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.413869 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.414048 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":274,"thread_start_us":118,"threads_started":1}
WARNING: no policy specified for kudu/127.31.43.253@KRBTEST.COM; defaulting to no policy
I20260812 04:00:07.415099 14831 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:00:07.415485 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.415370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:35567 (local address 127.0.0.1:52358)
0812 04:00:07.415418 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.415462 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:35567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 04:00:07.415638 14776 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.43.253:35567: connect: Connection refused (error 111)
I20260812 04:00:07.416136 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.416030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:35567 (local address 127.0.0.1:52370)
0812 04:00:07.416086 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.416116 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:35567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 04:00:07.416285 14831 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.253:35567: Network error: Client connection negotiation failed: client connection to 127.31.43.253:35567: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.253.keytab.
Entry for principal kudu/127.31.43.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.253.keytab.
I20260812 04:00:07.435657 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.253:35567
--webserver_interface=127.31.43.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.253.keytab
--principal=kudu/127.31.43.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:38165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:43117,127.31.43.253:35567,127.31.43.252:36551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:00:07.465586 14831 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } attempt: 1
I20260812 04:00:07.466140 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.465968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:35567 (local address 127.0.0.1:52380)
0812 04:00:07.466041 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.466095 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:35567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 04:00:07.466560 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.466459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:35567 (local address 127.0.0.1:52392)
0812 04:00:07.466506 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.466538 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:35567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 04:00:07.466717 14831 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.253:35567: Network error: Client connection negotiation failed: client connection to 127.31.43.253:35567: connect: Connection refused (error 111)
W20260812 04:00:07.522008 14839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:07.522230 14839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:07.522254 14839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:07.523530 14839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:00:07.523576 14839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:07.523590 14839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:07.523604 14839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:00:07.523617 14839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:00:07.525216 14839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:38165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:43117,127.31.43.253:35567,127.31.43.252:36551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.253.keytab
--principal=kudu/127.31.43.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.253:35567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.43.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:07.525497 14839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:07.525750 14839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:07.530876 14845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:07.530876 14848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:07.530941 14844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:07.531220 14839 server_base.cc:1034] running on GCE node
I20260812 04:00:07.531546 14839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:07.531870 14839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:07.533046 14839 hybrid_clock.cc:648] HybridClock initialized: now 1786507207533053 us; error 70 us; skew 500 ppm
I20260812 04:00:07.534886 14831 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } attempt: 2
I20260812 04:00:07.535327 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.535193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:35567 (local address 127.0.0.1:52404)
0812 04:00:07.535259 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.535294 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:35567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:07.535719 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.535616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:35567 (local address 127.0.0.1:52406)
0812 04:00:07.535665 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.535691 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:35567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 04:00:07.535842 14831 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.253:35567: Network error: Client connection negotiation failed: client connection to 127.31.43.253:35567: connect: Connection refused (error 111)
I20260812 04:00:07.536067 14839 init.cc:377] Logged in from keytab as kudu/127.31.43.253@KRBTEST.COM (short username kudu)
I20260812 04:00:07.536587 14839 webserver.cc:460] Webserver started at http://127.31.43.253:40075/ using document root <none> and password file <none>
I20260812 04:00:07.536827 14839 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:07.536873 14839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:07.537025 14839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:00:07.538134 14839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/data/instance:
uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3"
format_stamp: "Formatted at 2026-08-12 04:00:07 on dist-test-slave-8tdl"
server_key: "71f82865d752be222b072007b87c42ce"
server_key_iv: "33d82c7c25dbe5805b96741dc11b8df2"
server_key_version: "encryptionkey@0"
I20260812 04:00:07.538528 14839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/wal/instance:
uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3"
format_stamp: "Formatted at 2026-08-12 04:00:07 on dist-test-slave-8tdl"
server_key: "71f82865d752be222b072007b87c42ce"
server_key_iv: "33d82c7c25dbe5805b96741dc11b8df2"
server_key_version: "encryptionkey@0"
I20260812 04:00:07.540035 14839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:00:07.541010 14856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:07.541283 14839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:07.541388 14839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/wal
uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3"
format_stamp: "Formatted at 2026-08-12 04:00:07 on dist-test-slave-8tdl"
server_key: "71f82865d752be222b072007b87c42ce"
server_key_iv: "33d82c7c25dbe5805b96741dc11b8df2"
server_key_version: "encryptionkey@0"
I20260812 04:00:07.541472 14839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:07.560247 14839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:07.562436 14839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:07.562606 14839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:07.567083 14839 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.253:35567
I20260812 04:00:07.567144 14914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.253:35567 every 8 connection(s)
I20260812 04:00:07.567525 14839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
I20260812 04:00:07.568472 14915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:07.570140 14915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:00:07.572158 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 14839
I20260812 04:00:07.572280 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-1/wal/instance
I20260812 04:00:07.572479 31919 external_mini_cluster.cc:1428] Setting key 5bd2024ffd789408012d0a2d925668e4
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.253@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:07.580538 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.570908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47038 (local address 127.31.43.254:43117)
0812 04:00:07.571042 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.571044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.571196 (+ 152us) server_negotiation.cc:407] Connection header received
0812 04:00:07.571766 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.571767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.571790 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.571836 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.572765 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.573267 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.574754 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.575012 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.575101 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.577399 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.577401 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.577401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.577409 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.579045 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.579503 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.579505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.579505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.579566 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.579843 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.579844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.579844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.579971 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.580043 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.580341 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.580434 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 04:00:07.580806 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.570898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:43117 (local address 127.0.0.1:47038)
0812 04:00:07.571070 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.571114 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:07.571643 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:07.571919 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:07.571922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:07.572333 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:07.572655 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.572663 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.573369 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.573370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.574619 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.574622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.575111 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.575112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.575248 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.575849 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:07.575858 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:07.577299 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:07.579132 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.579140 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.579143 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.579401 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.579636 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.579637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.579638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.579765 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.580048 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:07.580056 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:07.580203 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 04:00:07.580309 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 04:00:07.580492 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.31.43.252@KRBTEST.COM; defaulting to no policy
I20260812 04:00:07.581365 14915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.252@KRBTEST.COM" created.
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.253@KRBTEST.COM for kudu/127.31.43.253@KRBTEST.COM
I20260812 04:00:07.588896 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.581667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:35567 (local address 127.0.0.1:52420)
0812 04:00:07.581717 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.581728 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:07.581770 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:07.582077 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:07.582078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:07.582198 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:07.582270 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.582271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.583163 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.583163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.583655 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.583656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.584230 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.584231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.584303 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.584775 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:07.584780 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:07.585971 (+ 1191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:07.587745 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.587746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.587746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.587911 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.588154 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.588155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.588155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.588216 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.588565 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:07.588566 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:07.588605 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 04:00:07.588672 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 04:00:07.588776 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 04:00:07.588982 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.581761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52420 (local address 127.31.43.253:35567)
0812 04:00:07.581907 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.581911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.581927 (+ 16us) server_negotiation.cc:407] Connection header received
0812 04:00:07.581942 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.581944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.581954 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.582013 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.582330 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.583087 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.583727 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.584096 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.584229 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.586063 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.586072 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.586072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.586079 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.587643 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.588013 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.588017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.588017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.588076 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.588278 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.588279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.588279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.588484 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.588590 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.588700 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.588850 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260812 04:00:07.589779 14915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:00:07.590179 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.590047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:32946)
0812 04:00:07.590097 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.590143 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 04:00:07.590348 14860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
I20260812 04:00:07.590888 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.590790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:32962)
0812 04:00:07.590837 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.590862 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 04:00:07.591161 14915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:36551: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.252.keytab.
Entry for principal kudu/127.31.43.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.252.keytab.
I20260812 04:00:07.601795 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.252:36551
--webserver_interface=127.31.43.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.252.keytab
--principal=kudu/127.31.43.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:38165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:43117,127.31.43.253:35567,127.31.43.252:36551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:00:07.627008 14831 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } attempt: 3
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.253@KRBTEST.COM
I20260812 04:00:07.635141 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.627451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:35567 (local address 127.0.0.1:52436)
0812 04:00:07.627518 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.627536 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:07.627613 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:07.627793 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:07.627793 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:07.628075 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:07.628153 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.628156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.628942 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.628943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.629541 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.629543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.629987 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.629988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.630079 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.630615 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:07.630621 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:07.632191 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:07.633967 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.633968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.633969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.634149 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.634371 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.634372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.634372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.634437 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.634743 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:07.634744 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:07.634806 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 04:00:07.634926 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 04:00:07.635036 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 04:00:07.635141 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.627538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52436 (local address 127.31.43.253:35567)
0812 04:00:07.627617 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.627619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.627631 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:00:07.627689 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.627690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.627702 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.627785 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.628259 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.628835 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.629645 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.629884 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.629985 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.632287 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.632289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.632289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.632300 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.633792 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.634231 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.634232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.634233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.634297 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.634504 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.634505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.634506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.634632 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.634752 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.634955 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.635037 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 04:00:07.635761 14831 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:00:07.636171 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.636053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:32968)
0812 04:00:07.636107 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.636136 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 04:00:07.636629 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.636486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:32980)
0812 04:00:07.636573 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.636605 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260812 04:00:07.636788 14831 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:36551: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
I20260812 04:00:07.640413 14915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } attempt: 1
I20260812 04:00:07.640861 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.640713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:32986)
0812 04:00:07.640792 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.640831 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 04:00:07.641309 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.641192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:32992)
0812 04:00:07.641249 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.641286 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 04:00:07.641455 14915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:36551: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
I20260812 04:00:07.667971 14831 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } attempt: 1
I20260812 04:00:07.668591 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.668407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:33008)
0812 04:00:07.668488 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.668546 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 04:00:07.669111 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.668974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:33020)
0812 04:00:07.669038 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.669073 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 04:00:07.669276 14831 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:36551: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
W20260812 04:00:07.691751 14922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:07.692023 14922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:07.692062 14922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:07.693233 14922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:00:07.693300 14922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:07.693323 14922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:07.693353 14922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:00:07.693385 14922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:00:07.694963 14922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:38165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:43117,127.31.43.253:35567,127.31.43.252:36551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.252.keytab
--principal=kudu/127.31.43.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.252:36551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.43.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:07.695251 14922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:07.695484 14922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:07.700507 14929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:07.700507 14932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:07.700559 14927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:07.701045 14922 server_base.cc:1034] running on GCE node
I20260812 04:00:07.701336 14922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:07.701643 14922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:07.702811 14922 hybrid_clock.cc:648] HybridClock initialized: now 1786507207702798 us; error 47 us; skew 500 ppm
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:07.705430 14922 init.cc:377] Logged in from keytab as kudu/127.31.43.252@KRBTEST.COM (short username kudu)
I20260812 04:00:07.705889 14922 webserver.cc:460] Webserver started at http://127.31.43.252:40793/ using document root <none> and password file <none>
I20260812 04:00:07.706172 14922 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:07.706251 14922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:07.706404 14922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:00:07.707576 14922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/data/instance:
uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8"
format_stamp: "Formatted at 2026-08-12 04:00:07 on dist-test-slave-8tdl"
server_key: "5d7f9d46cb7ba9515b8e079aed6b55fc"
server_key_iv: "86502e80223d7609f0eae6225dd2c868"
server_key_version: "encryptionkey@0"
I20260812 04:00:07.708000 14922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/wal/instance:
uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8"
format_stamp: "Formatted at 2026-08-12 04:00:07 on dist-test-slave-8tdl"
server_key: "5d7f9d46cb7ba9515b8e079aed6b55fc"
server_key_iv: "86502e80223d7609f0eae6225dd2c868"
server_key_version: "encryptionkey@0"
I20260812 04:00:07.709442 14922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 04:00:07.709604 14915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } attempt: 2
I20260812 04:00:07.710070 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.709919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:33032)
0812 04:00:07.709986 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.710031 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 04:00:07.710508 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.710368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:33042)
0812 04:00:07.710422 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.710476 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 04:00:07.710531 14939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260812 04:00:07.710641 14915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:36551: Network error: Client connection negotiation failed: client connection to 127.31.43.252:36551: connect: Connection refused (error 111)
I20260812 04:00:07.710744 14922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:00:07.710834 14922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/wal
uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8"
format_stamp: "Formatted at 2026-08-12 04:00:07 on dist-test-slave-8tdl"
server_key: "5d7f9d46cb7ba9515b8e079aed6b55fc"
server_key_iv: "86502e80223d7609f0eae6225dd2c868"
server_key_version: "encryptionkey@0"
I20260812 04:00:07.710912 14922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:07.730661 14922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:07.733284 14922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:07.733462 14922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:07.737677 14922 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.252:36551
I20260812 04:00:07.738044 14997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.252:36551 every 8 connection(s)
I20260812 04:00:07.738088 14922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
I20260812 04:00:07.739216 14998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:07.738382 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 14922
I20260812 04:00:07.739735 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786507041847525-31919-0/minicluster-data/master-2/wal/instance
I20260812 04:00:07.739924 31919 external_mini_cluster.cc:1428] Setting key 7755b76ce151837b71a42db0c7417fd6
I20260812 04:00:07.742169 14998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:00:07.744444 14831 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } attempt: 2
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.252@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.252@KRBTEST.COM
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.252@KRBTEST.COM
I20260812 04:00:07.752462 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.743064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47050 (local address 127.31.43.254:43117)
0812 04:00:07.743135 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.743137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.743303 (+ 166us) server_negotiation.cc:407] Connection header received
0812 04:00:07.743372 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.743373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.743384 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.743428 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.744084 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.744890 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.746022 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.746263 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.746346 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.748864 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.748866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.748866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.748874 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.750793 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.751240 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.751242 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.751242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.751310 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.751624 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.751625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.751626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.751848 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.751925 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.752267 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.752354 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 04:00:07.752853 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.742999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:43117 (local address 127.0.0.1:47050)
0812 04:00:07.743180 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.743225 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:07.743305 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:07.743456 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:07.743458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:07.743866 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:07.743953 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.743955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.744993 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.744994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.745903 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.745905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.746360 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.746362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.746438 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.746914 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:07.746925 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:07.748733 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:07.750898 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.750905 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.750907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.751155 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.751403 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.751404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.751404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.751555 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.751907 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:07.751915 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:07.752093 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 04:00:07.752238 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 04:00:07.752431 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":117,"threads_started":1}
I20260812 04:00:07.752997 15001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.740534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.31.43.252:36551)
0812 04:00:07.740754 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.740765 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.740806 (+ 41us) server_negotiation.cc:407] Connection header received
0812 04:00:07.741353 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.741363 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.741446 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.741621 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.742351 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.743891 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.745181 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.745760 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.745883 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.749376 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.749385 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.749385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.749394 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.751450 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.751884 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.751887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.751888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.751962 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.752195 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.752197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.752197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.752384 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.752531 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.752785 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.752868 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260812 04:00:07.753715 14998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:00:07.754010 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.744743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:33062)
0812 04:00:07.744791 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.744802 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:07.744842 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:07.745260 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:07.745261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:07.745433 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:07.745473 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.745475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.746402 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.746403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.747091 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.747094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.747462 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.747463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.747520 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.747897 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:07.747901 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:07.749763 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:07.752759 (+ 2996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.752760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.752760 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.752994 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.753331 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.753331 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.753332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.753417 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.753684 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:07.753685 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:07.753727 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 04:00:07.753817 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 04:00:07.753915 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 04:00:07.754036 15003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.744775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33062 (local address 127.31.43.252:36551)
0812 04:00:07.745045 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.745047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.745055 (+ 8us) server_negotiation.cc:407] Connection header received
0812 04:00:07.745071 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.745071 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.745081 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.745224 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.745573 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.746306 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.747173 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.747365 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.747455 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.749894 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.749895 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.749896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.749903 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.752664 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.753146 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.753147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.753147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.753236 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.753482 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.753483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.753484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.753622 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.753690 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.753843 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.753923 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":238,"thread_start_us":83,"threads_started":1}
I20260812 04:00:07.756021 14831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de: Bootstrap starting.
I20260812 04:00:07.756757 14831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:07.757090 14831 log.cc:824] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:07.758508 14831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de: No bootstrap required, opened a new log
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.252@KRBTEST.COM for kudu/127.31.43.253@KRBTEST.COM
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:07.760979 14831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } }
I20260812 04:00:07.761160 14831 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:07.761204 14831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d10f64a9854451497053ac6808752de, State: Initialized, Role: FOLLOWER
I20260812 04:00:07.761405 14831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } }
I20260812 04:00:07.761955 14831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:00:07.761934 15005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } } }
I20260812 04:00:07.762154 15005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:00:07.763274 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.754065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52448 (local address 127.31.43.253:35567)
0812 04:00:07.754123 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.754124 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.754562 (+ 438us) server_negotiation.cc:407] Connection header received
0812 04:00:07.754601 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.754602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.754614 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.754712 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.755042 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.755802 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.756747 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.757042 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.757118 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.759966 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.759968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.759969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.759978 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.761818 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.762248 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.762249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.762250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.762323 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.762552 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.762553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.762553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.762739 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.762793 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.763061 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.763158 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 04:00:07.763517 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.754430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47060 (local address 127.31.43.254:43117)
0812 04:00:07.754465 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.754467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.754475 (+ 8us) server_negotiation.cc:407] Connection header received
0812 04:00:07.754551 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.754552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.754562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.754605 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.755289 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.755749 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.756758 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.756963 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.757050 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.760128 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.760129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.760129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.760138 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.761987 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.762466 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.762467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.762468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.762533 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.762796 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.762797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.762798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.762942 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.763092 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.763296 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.763383 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 04:00:07.763964 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.753992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:35567 (local address 127.0.0.1:52448)
0812 04:00:07.754489 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.754499 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:07.754546 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:07.754705 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:07.754705 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:07.754882 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:07.754949 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.754951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.755879 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.755880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.756649 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.756651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.757181 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.757182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.757258 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.757796 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:07.757800 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:07.759849 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:07.761922 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.761924 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.761924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.762153 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.762419 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.762420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.762420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.762482 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.762852 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:07.762853 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:07.762908 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 04:00:07.763716 (+ 808us) client_negotiation.cc:240] Negotiation successful
0812 04:00:07.763869 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380}
I20260812 04:00:07.764408 14998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } has no permanent_uuid. Determining permanent_uuid...
W20260812 04:00:07.764266 15019 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:00:07.764892 15019 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:00:07.765877 14755 master_runner.cc:186] Error getting master registration for 127.31.43.253:35567: OK, instance_id { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" instance_seqno: 1786507207565744 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 04:00:07.769532 14755 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.253@KRBTEST.COM
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.252@KRBTEST.COM for kudu/127.31.43.252@KRBTEST.COM
I20260812 04:00:07.774017 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.764682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:33066)
0812 04:00:07.764748 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.764757 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:07.764791 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:07.765210 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:07.765211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:07.765385 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:07.765429 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.765431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.766392 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.766392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.767361 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.767365 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.768273 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.768274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.768319 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.768767 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:07.768773 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:07.770450 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:07.772724 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.772726 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.772726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.772959 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.773217 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.773218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.773218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.773289 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.773594 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:07.773596 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:07.773651 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 04:00:07.773731 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 04:00:07.773863 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 04:00:07.774400 15003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.764992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33066 (local address 127.31.43.252:36551)
0812 04:00:07.765037 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.765040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.765056 (+ 16us) server_negotiation.cc:407] Connection header received
0812 04:00:07.765101 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.765103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.765113 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.765171 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.765516 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.766289 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.767459 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.768147 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.768225 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.770570 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.770571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.770572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.770579 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.772611 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.773057 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.773059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.773060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.773129 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.773365 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.773366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.773366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.773511 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.773578 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.774220 (+ 642us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.774298 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 04:00:07.775205 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.764394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.31.43.253:35567)
0812 04:00:07.764438 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.764440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.764449 (+ 9us) server_negotiation.cc:407] Connection header received
0812 04:00:07.764470 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.764471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.764482 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.764537 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.765192 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.765920 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.766896 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.767163 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.767230 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.770716 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.770717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.770717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.770725 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.773278 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.773713 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.773714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.773714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.773780 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.774016 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.774017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.774018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.774155 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.774962 (+ 807us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.775009 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.775104 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 04:00:07.776263 14998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8: Bootstrap starting.
I20260812 04:00:07.776986 14998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:07.777311 14998 log.cc:824] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:07.778331 14998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8: No bootstrap required, opened a new log
I20260812 04:00:07.779222 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:00:07.780400 14998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } }
I20260812 04:00:07.780568 14998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:07.780620 14998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9a8b0b3cd44400bb285821e3cfb9d8, State: Initialized, Role: FOLLOWER
I20260812 04:00:07.780759 14998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } }
I20260812 04:00:07.781111 14998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:00:07.781088 15026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } } }
I20260812 04:00:07.781199 15026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 04:00:07.783442 15037 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:00:07.783546 15037 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:00:07.784567 14922 master_runner.cc:186] Error getting master registration for 127.31.43.253:35567: OK, instance_id { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" instance_seqno: 1786507207565744 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 04:00:07.787269 14922 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 04:00:07.795969 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.787506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.31.43.253:35567)
0812 04:00:07.787554 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.787556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.787857 (+ 301us) server_negotiation.cc:407] Connection header received
0812 04:00:07.787965 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.787966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.787977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.788008 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.788846 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.789613 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.790732 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.791029 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.791074 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.792919 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.792921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.792921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.792930 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.794581 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.795048 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.795050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.795050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.795112 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.795309 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.795310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.795310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.795452 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.795544 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.795764 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.795859 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 04:00:07.796200 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.787023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47072 (local address 127.31.43.254:43117)
0812 04:00:07.787069 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.787071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.787833 (+ 762us) server_negotiation.cc:407] Connection header received
0812 04:00:07.787956 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.787957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.787971 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.788007 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.788936 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.789817 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.791025 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.791349 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.791395 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.793155 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.793157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.793157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.793166 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.794908 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.795316 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.795317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.795318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.795377 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.795583 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.795583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.795584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.795733 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.795806 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.796021 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.796103 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 04:00:07.801826 14915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } attempt: 3
I20260812 04:00:07.804181 15003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.787685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33074 (local address 127.31.43.252:36551)
0812 04:00:07.787732 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.787734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.796507 (+ 8773us) server_negotiation.cc:407] Connection header received
0812 04:00:07.796632 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.796634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.796648 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.796681 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.797318 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.798071 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.799066 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.799391 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.799470 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.801043 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.801045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.801045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.801054 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.802849 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.803271 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.803273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.803273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.803330 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.803550 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.803551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.803551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.803681 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.803754 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.803983 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.804070 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 12 04:00:07 dist-test-slave-8tdl krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507207, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.253@KRBTEST.COM for kudu/127.31.43.252@KRBTEST.COM
I20260812 04:00:07.811640 15001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.802771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33090 (local address 127.31.43.252:36551)
0812 04:00:07.802813 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:07.802814 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:07.802822 (+ 8us) server_negotiation.cc:407] Connection header received
0812 04:00:07.802843 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:07.802844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:07.802853 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:07.802903 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:07.803262 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.804077 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.805355 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.805694 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.805763 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.807994 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:07.807996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:07.807996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:07.808005 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:07.809739 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.810758 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.810759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.810759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.810824 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:07.811065 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:07.811066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:07.811067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:07.811209 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:07.811278 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:07.811450 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 04:00:07.811539 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 04:00:07.811641 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:07.802467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:36551 (local address 127.0.0.1:33090)
0812 04:00:07.802562 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:07.802573 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:07.802623 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:07.802952 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:07.802953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:07.803129 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:07.803182 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.803184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.804179 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.804180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.805256 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:07.805260 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:07.805821 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:07.805822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:07.805885 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:07.806319 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:07.806325 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:07.807899 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:07.809835 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.809836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.809836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.810066 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.810906 (+ 840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:07.810907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:07.810907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:07.810985 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:07.811285 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:07.811286 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:07.811337 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 04:00:07.811413 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 04:00:07.811522 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 04:00:07.813906 14915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3: Bootstrap starting.
I20260812 04:00:07.814792 14915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:07.815128 14915 log.cc:824] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:07.816224 14915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3: No bootstrap required, opened a new log
I20260812 04:00:07.818361 14915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } }
I20260812 04:00:07.818521 14915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:07.818557 14915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cc4eb46a05b4fb6ad625cdaa78b9af3, State: Initialized, Role: FOLLOWER
I20260812 04:00:07.818775 14915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } }
I20260812 04:00:07.819169 14915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 04:00:07.821951 15055 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:00:07.822065 15055 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:00:07.819128 15044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } } }
I20260812 04:00:07.823433 15044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:00:07.823490 14839 master_runner.cc:263] No leader master found from master 0cc4eb46a05b4fb6ad625cdaa78b9af3
I20260812 04:00:07.823539 14839 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 04:00:07.895701 14778 tablet.cc:2369] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:00:07.926513 15005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:00:07.926653 15005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } }
I20260812 04:00:07.926918 15005 leader_election.cc:288] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cc4eb46a05b4fb6ad625cdaa78b9af3 (127.31.43.253:35567), 3f9a8b0b3cd44400bb285821e3cfb9d8 (127.31.43.252:36551)
I20260812 04:00:07.927304 14887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d10f64a9854451497053ac6808752de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" is_pre_election: true
I20260812 04:00:07.927476 14887 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d10f64a9854451497053ac6808752de in term 0.
I20260812 04:00:07.927650 14969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d10f64a9854451497053ac6808752de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" is_pre_election: true
I20260812 04:00:07.927817 14969 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d10f64a9854451497053ac6808752de in term 0.
I20260812 04:00:07.928288 14776 leader_election.cc:302] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cc4eb46a05b4fb6ad625cdaa78b9af3, 8d10f64a9854451497053ac6808752de; no voters:
I20260812 04:00:07.928550 15005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 04:00:07.928630 15005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 04:00:07.928653 15005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:07.929682 15005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } }
I20260812 04:00:07.929804 15005 leader_election.cc:288] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [CANDIDATE]: Term 1 election: Requested vote from peers 0cc4eb46a05b4fb6ad625cdaa78b9af3 (127.31.43.253:35567), 3f9a8b0b3cd44400bb285821e3cfb9d8 (127.31.43.252:36551)
I20260812 04:00:07.930168 14969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d10f64a9854451497053ac6808752de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8"
I20260812 04:00:07.930248 14969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:07.930773 14887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d10f64a9854451497053ac6808752de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3"
I20260812 04:00:07.930855 14887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:07.931718 14887 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d10f64a9854451497053ac6808752de in term 1.
I20260812 04:00:07.931951 14776 leader_election.cc:302] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cc4eb46a05b4fb6ad625cdaa78b9af3, 8d10f64a9854451497053ac6808752de; no voters:
I20260812 04:00:07.932502 14969 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d10f64a9854451497053ac6808752de in term 1.
I20260812 04:00:07.932762 15005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:00:07.932885 15005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [term 1 LEADER]: Becoming Leader. State: Replica: 8d10f64a9854451497053ac6808752de, State: Running, Role: LEADER
I20260812 04:00:07.932967 15005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } }
I20260812 04:00:07.933336 15005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d10f64a9854451497053ac6808752de. Latest consensus state: current_term: 1 leader_uuid: "8d10f64a9854451497053ac6808752de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } } }
I20260812 04:00:07.933394 15005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:07.933715 15059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:00:07.933945 15059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:00:07.936355 14969 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [term 1 FOLLOWER]: Refusing update from remote peer 8d10f64a9854451497053ac6808752de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:00:07.936627 14887 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [term 1 FOLLOWER]: Refusing update from remote peer 8d10f64a9854451497053ac6808752de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:00:07.937105 15005 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:00:07.939824 15005 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:00:07.941175 15026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d10f64a9854451497053ac6808752de. Latest consensus state: current_term: 1 leader_uuid: "8d10f64a9854451497053ac6808752de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } } }
I20260812 04:00:07.941303 15026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:00:07.941382 15044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d10f64a9854451497053ac6808752de. Latest consensus state: current_term: 1 leader_uuid: "8d10f64a9854451497053ac6808752de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } } }
I20260812 04:00:07.941507 15044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:00:07.942646 15005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8d10f64a9854451497053ac6808752de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } } }
I20260812 04:00:07.942704 15044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8d10f64a9854451497053ac6808752de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } } }
I20260812 04:00:07.942770 15005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:07.942778 15044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:00:07.943044 15060 mvcc.cc:205] Tried to move back new op lower bound from 7317533523704205312 to 7317533523694006272. Current Snapshot: MvccSnapshot[applied={T|T < 7317533523704205312}]
I20260812 04:00:07.943753 15005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8d10f64a9854451497053ac6808752de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } } }
I20260812 04:00:07.943840 15005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:07.944207 15026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8d10f64a9854451497053ac6808752de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d10f64a9854451497053ac6808752de" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43117 } } peers { permanent_uuid: "0cc4eb46a05b4fb6ad625cdaa78b9af3" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 35567 } } peers { permanent_uuid: "3f9a8b0b3cd44400bb285821e3cfb9d8" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 36551 } } }
I20260812 04:00:07.944293 15026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:00:07.945056 15061 mvcc.cc:205] Tried to move back new op lower bound from 7317533523704205312 to 7317533523694006272. Current Snapshot: MvccSnapshot[applied={T|T < 7317533523704205312}]
I20260812 04:00:07.945647 15059 catalog_manager.cc:1347] Generated new cluster ID: fdc42231c96a44d589ead67ab6be3231
I20260812 04:00:07.945729 15059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:00:07.971036 15059 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:00:07.971889 15059 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:00:07.982286 15059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d10f64a9854451497053ac6808752de: Generated new TSK 0
I20260812 04:00:07.982568 15059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:00:08.784164 15037 catalog_manager.cc:1259] Loaded cluster ID: fdc42231c96a44d589ead67ab6be3231
I20260812 04:00:08.784251 15037 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8: loading cluster ID for follower catalog manager: success
I20260812 04:00:08.785091 15037 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8: acquiring CA information for follower catalog manager: success
I20260812 04:00:08.785392 15037 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3f9a8b0b3cd44400bb285821e3cfb9d8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 04:00:08.822587 15055 catalog_manager.cc:1259] Loaded cluster ID: fdc42231c96a44d589ead67ab6be3231
I20260812 04:00:08.822695 15055 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3: loading cluster ID for follower catalog manager: success
I20260812 04:00:08.823470 15055 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3: acquiring CA information for follower catalog manager: success
I20260812 04:00:08.823729 15055 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0cc4eb46a05b4fb6ad625cdaa78b9af3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 04:00:09.038244 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 14755
I20260812 04:00:09.047247 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 14839
I20260812 04:00:09.054428 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 14922
2026-08-12T04:00:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3095 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:00:09 dist-test-slave-8tdl krb5kdc[15077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:00:09 dist-test-slave-8tdl krb5kdc[15077](info): set up 2 sockets
Aug 12 04:00:09 dist-test-slave-8tdl krb5kdc[15077](info): commencing operation
krb5kdc: starting...
W20260812 04:00:10.173897 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:00:10 dist-test-slave-8tdl krb5kdc[15077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:00:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:00:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:00:10.307976 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40063
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:40063
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:10.395089 15091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:10.395334 15091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:10.395373 15091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:10.396559 15091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:00:10.396631 15091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:10.396654 15091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:10.396680 15091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:00:10.396713 15091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:00:10.398252 15091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:40063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:10.398540 15091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:10.398850 15091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:10.404168 15098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:10.404173 15101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:10.404225 15099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:10.404460 15091 server_base.cc:1034] running on GCE node
I20260812 04:00:10.404816 15091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:10.405126 15091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:10.406298 15091 hybrid_clock.cc:648] HybridClock initialized: now 1786507210406284 us; error 44 us; skew 500 ppm
Aug 12 04:00:10 dist-test-slave-8tdl krb5kdc[15077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507210, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:10.409055 15091 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:00:10.409556 15091 webserver.cc:460] Webserver started at http://127.31.43.254:35295/ using document root <none> and password file <none>
I20260812 04:00:10.409824 15091 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:10.409899 15091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:10.410050 15091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:00:10.411304 15091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "22d7eeeef92b437cb9eef96cb1afdcf8"
format_stamp: "Formatted at 2026-08-12 04:00:10 on dist-test-slave-8tdl"
server_key: "1a6358ede41a7ad62027bd54e6c0c4fb"
server_key_iv: "59e18aa98bd70c96c0f4451a9fe726ac"
server_key_version: "encryptionkey@0"
I20260812 04:00:10.411674 15091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "22d7eeeef92b437cb9eef96cb1afdcf8"
format_stamp: "Formatted at 2026-08-12 04:00:10 on dist-test-slave-8tdl"
server_key: "1a6358ede41a7ad62027bd54e6c0c4fb"
server_key_iv: "59e18aa98bd70c96c0f4451a9fe726ac"
server_key_version: "encryptionkey@0"
I20260812 04:00:10.413148 15091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:00:10.414017 15108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:10.414217 15091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:10.414330 15091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "22d7eeeef92b437cb9eef96cb1afdcf8"
format_stamp: "Formatted at 2026-08-12 04:00:10 on dist-test-slave-8tdl"
server_key: "1a6358ede41a7ad62027bd54e6c0c4fb"
server_key_iv: "59e18aa98bd70c96c0f4451a9fe726ac"
server_key_version: "encryptionkey@0"
I20260812 04:00:10.414405 15091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:10.444741 15091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:10.447221 15091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:10.447403 15091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:10.451642 15091 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:40063
I20260812 04:00:10.451684 15174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:40063 every 8 connection(s)
I20260812 04:00:10.452090 15091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:00:10.453065 15175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:10.454672 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 15091
I20260812 04:00:10.454835 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:00:10.455041 31919 external_mini_cluster.cc:1428] Setting key 304972c7ce3050fc0a0d977ecceaeed1
I20260812 04:00:10.455775 15175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8: Bootstrap starting.
I20260812 04:00:10.456424 15175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:10.456804 15175 log.cc:824] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:10.457619 15175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8: No bootstrap required, opened a new log
I20260812 04:00:10.459853 15175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d7eeeef92b437cb9eef96cb1afdcf8" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40063 } }
I20260812 04:00:10.460072 15175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:10.460117 15175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d7eeeef92b437cb9eef96cb1afdcf8, State: Initialized, Role: FOLLOWER
I20260812 04:00:10.460281 15175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d7eeeef92b437cb9eef96cb1afdcf8" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40063 } }
I20260812 04:00:10.460371 15175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:00:10.460429 15175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:00:10.460486 15175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:10.461328 15175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d7eeeef92b437cb9eef96cb1afdcf8" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40063 } }
I20260812 04:00:10.461441 15175 leader_election.cc:302] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22d7eeeef92b437cb9eef96cb1afdcf8; no voters:
I20260812 04:00:10.461635 15175 leader_election.cc:288] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:00:10.461702 15180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:00:10.461869 15180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [term 1 LEADER]: Becoming Leader. State: Replica: 22d7eeeef92b437cb9eef96cb1afdcf8, State: Running, Role: LEADER
I20260812 04:00:10.461987 15175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:00:10.462023 15180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d7eeeef92b437cb9eef96cb1afdcf8" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40063 } }
Aug 12 04:00:10 dist-test-slave-8tdl krb5kdc[15077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:10.462405 15182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22d7eeeef92b437cb9eef96cb1afdcf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d7eeeef92b437cb9eef96cb1afdcf8" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40063 } } }
I20260812 04:00:10.462533 15182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:10.462438 15180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22d7eeeef92b437cb9eef96cb1afdcf8. Latest consensus state: current_term: 1 leader_uuid: "22d7eeeef92b437cb9eef96cb1afdcf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d7eeeef92b437cb9eef96cb1afdcf8" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40063 } } }
I20260812 04:00:10.462711 15180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8 [sys.catalog]: This master's current role is: LEADER
W20260812 04:00:10.463938 15198 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:00:10.464092 15198 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:00:10.464175 15188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:00:10.464372 15188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:00:10.466076 15188 catalog_manager.cc:1347] Generated new cluster ID: 8bba61bc9f614cd1be9a70dfec628878
I20260812 04:00:10.466132 15188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:00:10.466851 15178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:10.455612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57698 (local address 127.31.43.254:40063)
0812 04:00:10.455811 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:10.455819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:10.455844 (+ 25us) server_negotiation.cc:407] Connection header received
0812 04:00:10.456309 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:10.456315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:10.456361 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:10.456474 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:10.457339 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:10.458322 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:10.459433 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:10.459828 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:10.459904 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:10.462978 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:10.462983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:10.462983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:10.462995 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:10.464950 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:10.465379 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:10.465382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:10.465383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:10.465492 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:10.465732 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:10.465733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:10.465734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:10.465999 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:10.466185 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:10.466439 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 04:00:10.466595 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
I20260812 04:00:10.467797 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:00:10.479348 15188 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:00:10.479949 15188 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:00:10.489142 15188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22d7eeeef92b437cb9eef96cb1afdcf8: Generated new TSK 0
I20260812 04:00:10.489368 15188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:00:10.521816 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 15091
2026-08-12T04:00:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1465 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:00:10 dist-test-slave-8tdl krb5kdc[15205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:00:10 dist-test-slave-8tdl krb5kdc[15205](info): set up 2 sockets
Aug 12 04:00:10 dist-test-slave-8tdl krb5kdc[15205](info): commencing operation
krb5kdc: starting...
W20260812 04:00:11.631996 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:00:11 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:00:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:00:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:00:11.775424 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40475
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:41269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:40475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:11.863299 15220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:11.863590 15220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:11.863633 15220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:11.864938 15220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:00:11.865031 15220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:11.865057 15220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:11.865085 15220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:00:11.865116 15220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:00:11.866806 15220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:41269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:40475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:11.867139 15220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:11.867403 15220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:11.872934 15226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:11.872934 15230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:11.873142 15220 server_base.cc:1034] running on GCE node
W20260812 04:00:11.873065 15227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:11.873463 15220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:11.873811 15220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:11.875026 15220 hybrid_clock.cc:648] HybridClock initialized: now 1786507211874983 us; error 88 us; skew 500 ppm
Aug 12 04:00:11 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507211, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:11.885782 15220 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:00:11.886282 15220 webserver.cc:460] Webserver started at http://127.31.43.254:44615/ using document root <none> and password file <none>
I20260812 04:00:11.886530 15220 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:11.886580 15220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:11.886758 15220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:00:11.887928 15220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "abd36733c3fc40dab1f4ac8ee45cd026"
format_stamp: "Formatted at 2026-08-12 04:00:11 on dist-test-slave-8tdl"
server_key: "f7313cbf4a397f9069ad7330ff373fa9"
server_key_iv: "b4999656c794834780de18a622640ae0"
server_key_version: "encryptionkey@0"
I20260812 04:00:11.888262 15220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "abd36733c3fc40dab1f4ac8ee45cd026"
format_stamp: "Formatted at 2026-08-12 04:00:11 on dist-test-slave-8tdl"
server_key: "f7313cbf4a397f9069ad7330ff373fa9"
server_key_iv: "b4999656c794834780de18a622640ae0"
server_key_version: "encryptionkey@0"
I20260812 04:00:11.889783 15220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 04:00:11.890616 15237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:11.890856 15220 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:00:11.890964 15220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "abd36733c3fc40dab1f4ac8ee45cd026"
format_stamp: "Formatted at 2026-08-12 04:00:11 on dist-test-slave-8tdl"
server_key: "f7313cbf4a397f9069ad7330ff373fa9"
server_key_iv: "b4999656c794834780de18a622640ae0"
server_key_version: "encryptionkey@0"
I20260812 04:00:11.891027 15220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:11.904697 15220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:11.907156 15220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:11.907285 15220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:11.911535 15220 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:40475
I20260812 04:00:11.911572 15296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:40475 every 8 connection(s)
I20260812 04:00:11.911965 15220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:00:11.912441 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 15220
I20260812 04:00:11.912546 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:00:11.912768 31919 external_mini_cluster.cc:1428] Setting key dd1b1695601355ba4387591ad51d1583
I20260812 04:00:11.913281 15297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:11.916519 15297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Bootstrap starting.
I20260812 04:00:11.917188 15297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:11.917513 15297 log.cc:824] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:11.918596 15297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: No bootstrap required, opened a new log
Aug 12 04:00:11 dist-test-slave-8tdl krb5kdc[15205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:11.921254 15297 raft_consensus.cc:348] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } }
I20260812 04:00:11.921458 15297 raft_consensus.cc:374] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:11.921489 15297 raft_consensus.cc:729] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd36733c3fc40dab1f4ac8ee45cd026, State: Initialized, Role: FOLLOWER
I20260812 04:00:11.921643 15297 consensus_queue.cc:260] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } }
I20260812 04:00:11.921743 15297 raft_consensus.cc:388] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:00:11.921813 15297 raft_consensus.cc:482] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:00:11.921862 15297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:11.922698 15297 raft_consensus.cc:504] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } }
I20260812 04:00:11.922821 15297 leader_election.cc:302] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abd36733c3fc40dab1f4ac8ee45cd026; no voters:
I20260812 04:00:11.923096 15297 leader_election.cc:288] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:00:11.923168 15304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:00:11.923416 15297 sys_catalog.cc:564] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:00:11.923547 15304 raft_consensus.cc:686] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 1 LEADER]: Becoming Leader. State: Replica: abd36733c3fc40dab1f4ac8ee45cd026, State: Running, Role: LEADER
I20260812 04:00:11.923673 15304 consensus_queue.cc:237] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } }
I20260812 04:00:11.924135 15304 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abd36733c3fc40dab1f4ac8ee45cd026. Latest consensus state: current_term: 1 leader_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } } }
I20260812 04:00:11.924147 15305 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } } }
I20260812 04:00:11.924293 15305 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:11.924505 15304 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:11.924702 15314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:00:11.925385 15314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:00:11.925467 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:11.913365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42510 (local address 127.31.43.254:40475)
0812 04:00:11.913593 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:11.913601 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:11.913627 (+ 26us) server_negotiation.cc:407] Connection header received
0812 04:00:11.914107 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:11.914114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:11.914161 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:11.914320 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:11.915110 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:11.916029 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:11.916874 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:11.917355 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:11.917479 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:11.920421 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:11.920426 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:11.920427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:11.920443 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:11.922824 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:11.923330 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:11.923333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:11.923334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:11.923486 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:11.923739 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:11.923741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:11.923741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:11.924243 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:11.924418 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:11.924989 (+ 571us) server_negotiation.cc:299] Negotiation successful
0812 04:00:11.925251 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":114,"threads_started":1}
I20260812 04:00:11.927421 15314 catalog_manager.cc:1347] Generated new cluster ID: 7d5a162f7196494a9ce5dde02431a856
I20260812 04:00:11.927482 15314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:00:11.950913 15314 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:00:11.951747 15314 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:00:11.961386 15314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Generated new TSK 0
I20260812 04:00:11.961663 15314 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 04:00:12.003007 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40475
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:41269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:12.090569 15329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:12.090824 15329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:12.090849 15329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:12.091979 15329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:12.092028 15329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:12.092056 15329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:00:12.093611 15329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:41269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:12.093816 15329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:12.094154 15329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:12.099516 15337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:12.099561 15334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:12.099563 15335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:12.099833 15329 server_base.cc:1034] running on GCE node
I20260812 04:00:12.100036 15329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:12.100273 15329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:12.101425 15329 hybrid_clock.cc:648] HybridClock initialized: now 1786507212101410 us; error 33 us; skew 500 ppm
Aug 12 04:00:12 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507212, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:12.104280 15329 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 04:00:12.104771 15329 webserver.cc:460] Webserver started at http://127.31.43.193:37235/ using document root <none> and password file <none>
I20260812 04:00:12.105022 15329 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:12.105098 15329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:12.105252 15329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:00:12.106380 15329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "c4fed6b5c0a43a82c7d0d91d62165a39"
server_key_iv: "975d105db9bc5886788c448d070e7964"
server_key_version: "encryptionkey@0"
I20260812 04:00:12.106832 15329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "c4fed6b5c0a43a82c7d0d91d62165a39"
server_key_iv: "975d105db9bc5886788c448d070e7964"
server_key_version: "encryptionkey@0"
I20260812 04:00:12.108265 15329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:00:12.109097 15344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:12.109310 15329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:00:12.109439 15329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "c4fed6b5c0a43a82c7d0d91d62165a39"
server_key_iv: "975d105db9bc5886788c448d070e7964"
server_key_version: "encryptionkey@0"
I20260812 04:00:12.109524 15329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:12.132089 15329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:12.134932 15329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:12.135128 15329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:12.135418 15329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:12.135795 15329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:12.135859 15329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:12.135938 15329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:12.135974 15329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:12.142813 15329 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:36067
I20260812 04:00:12.142897 15472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:36067 every 8 connection(s)
I20260812 04:00:12.143287 15329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 04:00:12.149686 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 15329
I20260812 04:00:12.149828 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
Aug 12 04:00:12 dist-test-slave-8tdl krb5kdc[15205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507212, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:12.150032 31919 external_mini_cluster.cc:1428] Setting key eed4fc9fea8e10a8edfaf337483c7013
I20260812 04:00:12.153992 15475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.144072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:53039)
0812 04:00:12.144264 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:12.144313 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:12.144881 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:12.145140 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:12.145144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:12.145789 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:12.146010 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.146016 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.146761 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.146762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.147696 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.147700 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.148083 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.148084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.148203 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.148720 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:12.148729 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:12.150527 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:12.152388 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:12.152397 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:12.152399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:12.152761 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:12.153041 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:12.153042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:12.153042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:12.153155 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:12.153467 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:12.153472 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:12.153604 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 04:00:12.153688 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 04:00:12.153817 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260812 04:00:12.154011 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.144126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:53039 (local address 127.31.43.254:40475)
0812 04:00:12.144185 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:12.144187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:12.144412 (+ 225us) server_negotiation.cc:407] Connection header received
0812 04:00:12.144978 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:12.144979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:12.144989 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:12.145056 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:12.146105 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.146622 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.147786 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.147996 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.148050 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.150624 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:12.150626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:12.150626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:12.150635 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:12.152295 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:12.152857 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:12.152859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:12.152859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:12.152956 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:12.153228 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:12.153228 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:12.153229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:12.153396 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:12.153475 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:12.153740 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 04:00:12.153859 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 04:00:12.154913 15473 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40475
I20260812 04:00:12.155059 15473 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:12.155275 15473 heartbeater.cc:511] Master 127.31.43.254:40475 requested a full tablet report, sending...
I20260812 04:00:12.155983 15257 ts_manager.cc:194] Registered new tserver with Master: 1c8af0f13ad1411c9c3bc1553a4eb1cd (127.31.43.193:36067)
I20260812 04:00:12.157106 15257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:53039
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 04:00:12.181435 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40475
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:41269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:12.266942 15478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:12.267223 15478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:12.267269 15478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:12.268550 15478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:12.268627 15478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:12.268675 15478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:00:12.270336 15478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:41269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:12.270634 15478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:12.270924 15478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:12.276197 15484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:12.276175 15483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:12.276264 15478 server_base.cc:1034] running on GCE node
W20260812 04:00:12.276175 15486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:12.276676 15478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:12.276912 15478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:12.278095 15478 hybrid_clock.cc:648] HybridClock initialized: now 1786507212278071 us; error 41 us; skew 500 ppm
Aug 12 04:00:12 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507212, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:12.280833 15478 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 04:00:12.281293 15478 webserver.cc:460] Webserver started at http://127.31.43.194:44717/ using document root <none> and password file <none>
I20260812 04:00:12.281484 15478 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:12.281531 15478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:12.281629 15478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:00:12.282788 15478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "b528ed2b2162480a8bbcf21edfc18f6f"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "fc877be0679493e555011a3854dad37f"
server_key_iv: "3f8d72854995e68c1ae72bb4591452a7"
server_key_version: "encryptionkey@0"
I20260812 04:00:12.283130 15478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "b528ed2b2162480a8bbcf21edfc18f6f"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "fc877be0679493e555011a3854dad37f"
server_key_iv: "3f8d72854995e68c1ae72bb4591452a7"
server_key_version: "encryptionkey@0"
I20260812 04:00:12.284601 15478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 04:00:12.285413 15493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:12.285597 15478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:00:12.285691 15478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "b528ed2b2162480a8bbcf21edfc18f6f"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "fc877be0679493e555011a3854dad37f"
server_key_iv: "3f8d72854995e68c1ae72bb4591452a7"
server_key_version: "encryptionkey@0"
I20260812 04:00:12.285751 15478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:12.311213 15478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:12.313637 15478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:12.313769 15478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:12.313978 15478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:12.314339 15478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:12.314375 15478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:12.314422 15478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:12.314450 15478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:12.320808 15478 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:41077
I20260812 04:00:12.320851 15622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:41077 every 8 connection(s)
I20260812 04:00:12.321214 15478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:00:12.327827 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 15478
I20260812 04:00:12.327951 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 04:00:12.328161 31919 external_mini_cluster.cc:1428] Setting key d6ad51ca4dbeb9cf7f2b30127ef0f955
Aug 12 04:00:12 dist-test-slave-8tdl krb5kdc[15205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507212, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:12.332974 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.322070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:35643 (local address 127.31.43.254:40475)
0812 04:00:12.322170 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:12.322172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:12.322285 (+ 113us) server_negotiation.cc:407] Connection header received
0812 04:00:12.322805 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:12.322806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:12.322817 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:12.322856 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:12.323908 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.324772 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.326036 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.326291 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.326356 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.329452 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:12.329454 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:12.329454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:12.329462 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:12.331238 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:12.331801 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:12.331802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:12.331803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:12.331864 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:12.332291 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:12.332292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:12.332292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:12.332409 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:12.332473 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:12.332829 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 04:00:12.332906 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 04:00:12.333287 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.321988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:35643)
0812 04:00:12.322166 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:12.322204 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:12.322716 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:12.322917 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:12.322920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:12.323616 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:12.323821 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.323826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.324885 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.324886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.325881 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.325884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.326374 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.326374 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.326475 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.327191 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:12.327200 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:12.329334 (+ 2134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:12.331325 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:12.331335 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:12.331337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:12.331704 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:12.331944 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:12.331945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:12.331945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:12.332106 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:12.332488 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:12.332496 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:12.332687 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 04:00:12.332808 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 04:00:12.333005 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260812 04:00:12.333837 15623 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40475
I20260812 04:00:12.333950 15623 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:12.334177 15623 heartbeater.cc:511] Master 127.31.43.254:40475 requested a full tablet report, sending...
I20260812 04:00:12.334632 15257 ts_manager.cc:194] Registered new tserver with Master: b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077)
I20260812 04:00:12.335064 15257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:35643
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 04:00:12.358067 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40475
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:41269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:12.446949 15629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:12.447175 15629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:12.447199 15629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:12.448410 15629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:12.448457 15629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:12.448484 15629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:00:12.449981 15629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:41269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:12.450259 15629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:12.450507 15629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:12.455909 15635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:12.455909 15636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:12.456065 15629 server_base.cc:1034] running on GCE node
W20260812 04:00:12.455932 15639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:12.456432 15629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:12.456658 15629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:12.457813 15629 hybrid_clock.cc:648] HybridClock initialized: now 1786507212457792 us; error 39 us; skew 500 ppm
Aug 12 04:00:12 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507212, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:12.460559 15629 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 04:00:12.460989 15629 webserver.cc:460] Webserver started at http://127.31.43.195:44669/ using document root <none> and password file <none>
I20260812 04:00:12.461297 15629 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:12.461377 15629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:12.461537 15629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:00:12.462749 15629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "637df5a1285446a29ffcc32ea70051a7"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "7d3f7f5340b93984e8bd19341b80b5b0"
server_key_iv: "d591e86cc16a775eeece604ee8f04f19"
server_key_version: "encryptionkey@0"
I20260812 04:00:12.463165 15629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "637df5a1285446a29ffcc32ea70051a7"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "7d3f7f5340b93984e8bd19341b80b5b0"
server_key_iv: "d591e86cc16a775eeece604ee8f04f19"
server_key_version: "encryptionkey@0"
I20260812 04:00:12.464685 15629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 04:00:12.465651 15647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:12.465884 15629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:00:12.466007 15629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "637df5a1285446a29ffcc32ea70051a7"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "7d3f7f5340b93984e8bd19341b80b5b0"
server_key_iv: "d591e86cc16a775eeece604ee8f04f19"
server_key_version: "encryptionkey@0"
I20260812 04:00:12.466125 15629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:12.482632 15629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:12.485209 15629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:12.485399 15629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:12.485672 15629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:12.486016 15629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:12.486080 15629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:12.486119 15629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:12.486166 15629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:12.492645 15629 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:46713
I20260812 04:00:12.492691 15771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:46713 every 8 connection(s)
I20260812 04:00:12.493083 15629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 04:00:12.494902 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 15629
I20260812 04:00:12.495003 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 04:00:12.495190 31919 external_mini_cluster.cc:1428] Setting key 571555796a9313aec297331e31aa9f9a
Aug 12 04:00:12 dist-test-slave-8tdl krb5kdc[15205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507212, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:12.503093 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.493874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:33915 (local address 127.31.43.254:40475)
0812 04:00:12.493941 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:12.493943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:12.494159 (+ 216us) server_negotiation.cc:407] Connection header received
0812 04:00:12.494745 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:12.494746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:12.494757 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:12.494810 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:12.496344 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.496902 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.497932 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.498127 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.498189 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.500499 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:12.500501 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:12.500501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:12.500515 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:12.501725 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:12.502126 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:12.502126 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:12.502127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:12.502179 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:12.502431 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:12.502433 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:12.502433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:12.502558 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:12.502626 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:12.502944 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 04:00:12.503023 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 04:00:12.503252 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.493840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:33915)
0812 04:00:12.494017 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:12.494067 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:12.494613 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:12.494908 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:12.494912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:12.495893 (+ 981us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:12.496209 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.496219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.496999 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.497000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.497837 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.497839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.498211 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.498211 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.498316 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.498938 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:12.498947 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:12.500413 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:12.501800 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:12.501807 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:12.501808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:12.502029 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:12.502241 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:12.502242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:12.502242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:12.502345 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:12.502630 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:12.502635 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:12.502804 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 04:00:12.502892 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 04:00:12.503022 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260812 04:00:12.503777 15773 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40475
I20260812 04:00:12.503898 15773 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:12.504084 15773 heartbeater.cc:511] Master 127.31.43.254:40475 requested a full tablet report, sending...
I20260812 04:00:12.504495 15257 ts_manager.cc:194] Registered new tserver with Master: 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713)
I20260812 04:00:12.504925 15257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:33915
I20260812 04:00:12.507005 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 04:00:12.514437 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.508430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42512 (local address 127.31.43.254:40475)
0812 04:00:12.508492 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:12.508493 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:12.508628 (+ 135us) server_negotiation.cc:407] Connection header received
0812 04:00:12.508667 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:12.508668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:12.508677 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:12.508727 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:12.509303 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.509772 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.510483 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.510671 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.510780 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.511904 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:12.511905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:12.511905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:12.511912 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:12.513436 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:12.513750 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:12.513751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:12.513751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:12.513804 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:12.513967 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:12.513968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:12.513968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:12.514138 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:12.514221 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:12.514311 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 04:00:12.514371 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 04:00:12.516273 15257 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 04:00:12.518010 15257 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 04:00:12.525707 15791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.520648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:46713 (local address 127.0.0.1:35932)
0812 04:00:12.521009 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:12.521053 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:12.521137 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:12.521505 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:12.521508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:12.521509 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:12.521585 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.521587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.522979 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.522987 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.524231 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.524233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.525331 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.525333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.525426 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.525440 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 04:00:12.525540 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 04:00:12.525615 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":56,"threads_started":1}
I20260812 04:00:12.525712 15789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.520772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35932 (local address 127.31.43.195:46713)
0812 04:00:12.521009 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:12.521016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:12.521248 (+ 232us) server_negotiation.cc:407] Connection header received
0812 04:00:12.521299 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:12.521301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:12.521319 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:12.521408 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:12.521679 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.522864 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.524363 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.525218 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.525318 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.525386 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:12.525565 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 04:00:12.525611 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":109,"threads_started":1}
I20260812 04:00:12.526132 15787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.520855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:41077 (local address 127.0.0.1:40408)
0812 04:00:12.521197 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:12.521206 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:12.521258 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:12.521539 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:12.521540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:12.521543 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:12.521608 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.521610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.522602 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.522603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.524221 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.524225 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.525540 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.525541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.525613 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.525617 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:00:12.525925 (+ 308us) client_negotiation.cc:240] Negotiation successful
0812 04:00:12.526052 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":84,"threads_started":1}
I20260812 04:00:12.526427 15792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.520975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40408 (local address 127.31.43.194:41077)
0812 04:00:12.521187 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:12.521195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:12.521258 (+ 63us) server_negotiation.cc:407] Connection header received
0812 04:00:12.521327 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:12.521330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:12.521350 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:12.521438 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:12.521707 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.522484 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.524363 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.525373 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.525438 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.525485 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:12.526300 (+ 815us) server_negotiation.cc:299] Negotiation successful
0812 04:00:12.526348 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":133,"threads_started":1}
I20260812 04:00:12.526620 15696 tablet_service.cc:1467] Processing CreateTablet for tablet a74f5949113644d29371a59d2933be84 (DEFAULT_TABLE table=test-table [id=6810587146c140f6a79772ef03a3ce9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:12.527024 15696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a74f5949113644d29371a59d2933be84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:12.527151 15547 tablet_service.cc:1467] Processing CreateTablet for tablet a74f5949113644d29371a59d2933be84 (DEFAULT_TABLE table=test-table [id=6810587146c140f6a79772ef03a3ce9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:12.527379 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.520655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:36067 (local address 127.0.0.1:35676)
0812 04:00:12.521065 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:12.521077 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:12.521312 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:12.521590 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:12.521591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:12.521593 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:12.521646 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.521648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.524340 (+ 2692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.524341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.525492 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.525494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.527100 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.527101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.527177 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.527181 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:00:12.527246 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 04:00:12.527297 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":108,"threads_started":1}
I20260812 04:00:12.527575 15547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a74f5949113644d29371a59d2933be84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:12.527712 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.520763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35676 (local address 127.31.43.193:36067)
0812 04:00:12.521023 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:12.521027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:12.521137 (+ 110us) server_negotiation.cc:407] Connection header received
0812 04:00:12.521413 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:12.521415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:12.521432 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:12.521487 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:12.521745 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.522863 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.525603 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.526900 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.526955 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.527024 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:12.527270 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 04:00:12.527324 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":111,"threads_started":1}
I20260812 04:00:12.530092 15793 tablet_bootstrap.cc:492] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7: Bootstrap starting.
I20260812 04:00:12.530418 15794 tablet_bootstrap.cc:492] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f: Bootstrap starting.
I20260812 04:00:12.531033 15793 tablet_bootstrap.cc:654] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:12.531118 15794 tablet_bootstrap.cc:654] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:12.531343 15396 tablet_service.cc:1467] Processing CreateTablet for tablet a74f5949113644d29371a59d2933be84 (DEFAULT_TABLE table=test-table [id=6810587146c140f6a79772ef03a3ce9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:12.531715 15794 log.cc:824] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:12.531698 15793 log.cc:824] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:12.531749 15396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a74f5949113644d29371a59d2933be84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:12.532619 15794 tablet_bootstrap.cc:492] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f: No bootstrap required, opened a new log
I20260812 04:00:12.532696 15794 ts_tablet_manager.cc:1397] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 04:00:12.535065 15794 raft_consensus.cc:348] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } } peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } }
I20260812 04:00:12.535271 15794 raft_consensus.cc:374] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:12.535319 15794 raft_consensus.cc:729] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b528ed2b2162480a8bbcf21edfc18f6f, State: Initialized, Role: FOLLOWER
I20260812 04:00:12.535506 15794 consensus_queue.cc:260] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } } peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } }
I20260812 04:00:12.535864 15794 ts_tablet_manager.cc:1428] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260812 04:00:12.535921 15623 heartbeater.cc:503] Master 127.31.43.254:40475 was elected leader, sending a full tablet report...
I20260812 04:00:12.536383 15793 tablet_bootstrap.cc:492] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7: No bootstrap required, opened a new log
I20260812 04:00:12.536476 15793 ts_tablet_manager.cc:1397] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 04:00:12.537644 15799 tablet_bootstrap.cc:492] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Bootstrap starting.
I20260812 04:00:12.538445 15799 tablet_bootstrap.cc:654] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:12.539149 15799 log.cc:824] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:12.539922 15799 tablet_bootstrap.cc:492] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: No bootstrap required, opened a new log
I20260812 04:00:12.540017 15799 ts_tablet_manager.cc:1397] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 04:00:12.539108 15793 raft_consensus.cc:348] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } } peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } }
I20260812 04:00:12.540844 15793 raft_consensus.cc:374] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:12.540915 15793 raft_consensus.cc:729] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 637df5a1285446a29ffcc32ea70051a7, State: Initialized, Role: FOLLOWER
I20260812 04:00:12.541112 15793 consensus_queue.cc:260] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } } peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } }
I20260812 04:00:12.541472 15793 ts_tablet_manager.cc:1428] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 04:00:12.541666 15773 heartbeater.cc:503] Master 127.31.43.254:40475 was elected leader, sending a full tablet report...
I20260812 04:00:12.541981 15799 raft_consensus.cc:348] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } } peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } }
I20260812 04:00:12.542155 15799 raft_consensus.cc:374] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:12.542204 15799 raft_consensus.cc:729] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c8af0f13ad1411c9c3bc1553a4eb1cd, State: Initialized, Role: FOLLOWER
I20260812 04:00:12.542328 15799 consensus_queue.cc:260] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } } peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } }
I20260812 04:00:12.542593 15799 ts_tablet_manager.cc:1428] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 04:00:12.542717 15473 heartbeater.cc:503] Master 127.31.43.254:40475 was elected leader, sending a full tablet report...
I20260812 04:00:12.552809 15798 raft_consensus.cc:482] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:00:12.552983 15798 raft_consensus.cc:504] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } } peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } }
I20260812 04:00:12.553457 15798 leader_election.cc:288] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c8af0f13ad1411c9c3bc1553a4eb1cd (127.31.43.193:36067), 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713)
I20260812 04:00:12.557688 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.553807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:48573 (local address 127.31.43.193:36067)
0812 04:00:12.553856 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:12.553858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:12.553868 (+ 10us) server_negotiation.cc:407] Connection header received
0812 04:00:12.553901 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:12.553902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:12.553912 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:12.553986 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:12.554211 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.555155 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.556635 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.557306 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.557384 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.557397 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:12.557563 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 04:00:12.557608 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 04:00:12.557688 15789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.553748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:47101 (local address 127.31.43.195:46713)
0812 04:00:12.553798 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:12.553801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:12.554183 (+ 382us) server_negotiation.cc:407] Connection header received
0812 04:00:12.554222 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:12.554223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:12.554236 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:12.554297 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:12.554449 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.555292 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.556635 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.557288 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.557389 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.557397 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:12.557562 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 04:00:12.557609 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 04:00:12.557706 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.553651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:36067 (local address 127.31.43.194:48573)
0812 04:00:12.553704 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:12.553718 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:12.553780 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:12.554007 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:12.554008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:12.554011 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:12.554119 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.554123 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.555252 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.555253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.556512 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.556515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.557389 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.557390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.557476 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.557478 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 04:00:12.557539 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 04:00:12.557593 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":5}
I20260812 04:00:12.557717 15803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.553651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:46713 (local address 127.31.43.194:47101)
0812 04:00:12.554097 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:12.554108 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:12.554170 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:12.554301 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:12.554302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:12.554303 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:12.554371 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.554372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.555373 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.555374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.556512 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.556515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.557389 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.557390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.557473 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.557478 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 04:00:12.557539 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 04:00:12.557584 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":195,"thread_start_us":157,"threads_started":1}
I20260812 04:00:12.558068 15419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a74f5949113644d29371a59d2933be84" candidate_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" is_pre_election: true
I20260812 04:00:12.558072 15717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a74f5949113644d29371a59d2933be84" candidate_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "637df5a1285446a29ffcc32ea70051a7" is_pre_election: true
I20260812 04:00:12.558243 15717 raft_consensus.cc:2454] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b528ed2b2162480a8bbcf21edfc18f6f in term 0.
I20260812 04:00:12.558243 15419 raft_consensus.cc:2454] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b528ed2b2162480a8bbcf21edfc18f6f in term 0.
I20260812 04:00:12.558525 15497 leader_election.cc:302] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 637df5a1285446a29ffcc32ea70051a7, b528ed2b2162480a8bbcf21edfc18f6f; no voters:
I20260812 04:00:12.558661 15798 raft_consensus.cc:2781] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 04:00:12.558808 15798 raft_consensus.cc:482] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 04:00:12.558828 15798 raft_consensus.cc:3037] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:12.559697 15798 raft_consensus.cc:504] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } } peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } }
I20260812 04:00:12.559842 15798 leader_election.cc:288] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [CANDIDATE]: Term 1 election: Requested vote from peers 1c8af0f13ad1411c9c3bc1553a4eb1cd (127.31.43.193:36067), 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713)
I20260812 04:00:12.560014 15419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a74f5949113644d29371a59d2933be84" candidate_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd"
I20260812 04:00:12.560009 15717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a74f5949113644d29371a59d2933be84" candidate_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "637df5a1285446a29ffcc32ea70051a7"
I20260812 04:00:12.560103 15717 raft_consensus.cc:3037] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:12.560103 15419 raft_consensus.cc:3037] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:12.561033 15717 raft_consensus.cc:2454] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b528ed2b2162480a8bbcf21edfc18f6f in term 1.
I20260812 04:00:12.561085 15419 raft_consensus.cc:2454] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b528ed2b2162480a8bbcf21edfc18f6f in term 1.
I20260812 04:00:12.561213 15497 leader_election.cc:302] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 637df5a1285446a29ffcc32ea70051a7, b528ed2b2162480a8bbcf21edfc18f6f; no voters:
I20260812 04:00:12.561368 15798 raft_consensus.cc:2781] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:00:12.561527 15798 raft_consensus.cc:686] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 1 LEADER]: Becoming Leader. State: Replica: b528ed2b2162480a8bbcf21edfc18f6f, State: Running, Role: LEADER
I20260812 04:00:12.561640 15798 consensus_queue.cc:237] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } } peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } }
I20260812 04:00:12.562526 15257 catalog_manager.cc:5495] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f reported cstate change: term changed from 0 to 1, leader changed from <none> to b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194). New cstate: current_term: 1 leader_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } health_report { overall_health: HEALTHY } } }
W20260812 04:00:12.571764 15624 tablet.cc:2369] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:00:12.573617 15792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.571093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.31.43.194:41077)
0812 04:00:12.571135 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:12.571137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:12.571146 (+ 9us) server_negotiation.cc:407] Connection header received
0812 04:00:12.571233 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:12.571234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:12.571250 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:12.571325 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:00:12.571485 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.572018 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.572811 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.573003 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.573081 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.573273 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:00:12.573369 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:00:12.573463 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:12.573483 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 04:00:12.573508 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 04:00:12.575589 15717 raft_consensus.cc:1261] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7 [term 1 FOLLOWER]: Refusing update from remote peer b528ed2b2162480a8bbcf21edfc18f6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:00:12.575606 15419 raft_consensus.cc:1261] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Refusing update from remote peer b528ed2b2162480a8bbcf21edfc18f6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:00:12.575989 15804 consensus_queue.cc:1035] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:00:12.576144 15798 consensus_queue.cc:1035] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:00:12.579022 15807 mvcc.cc:205] Tried to move back new op lower bound from 7317533542706569216 to 7317533542653124608. Current Snapshot: MvccSnapshot[applied={T|T < 7317533542706569216}]
I20260812 04:00:12.579942 15808 mvcc.cc:205] Tried to move back new op lower bound from 7317533542706569216 to 7317533542653124608. Current Snapshot: MvccSnapshot[applied={T|T < 7317533542706569216}]
I20260812 04:00:12.581650 15257 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:00:12.581773 15257 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:00:12.582612 15257 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Sending DeleteTablet for 3 replicas of tablet a74f5949113644d29371a59d2933be84
I20260812 04:00:12.583019 15696 tablet_service.cc:1514] Processing DeleteTablet for tablet a74f5949113644d29371a59d2933be84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:12 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:35932
I20260812 04:00:12.583052 15547 tablet_service.cc:1514] Processing DeleteTablet for tablet a74f5949113644d29371a59d2933be84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:12 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:40408
I20260812 04:00:12.583272 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 15329
I20260812 04:00:12.583313 15817 tablet_replica.cc:330] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f: stopping tablet replica
I20260812 04:00:12.585978 15817 raft_consensus.cc:2229] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 1 LEADER]: Raft consensus shutting down.
I20260812 04:00:12.586097 15817 raft_consensus.cc:2258] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:00:12.587005 15817 ts_tablet_manager.cc:1905] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:00:12.588677 15817 ts_tablet_manager.cc:1918] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:00:12.588832 15817 log.cc:1196] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/a74f5949113644d29371a59d2933be84
I20260812 04:00:12.589213 15817 ts_tablet_manager.cc:1939] T a74f5949113644d29371a59d2933be84 P b528ed2b2162480a8bbcf21edfc18f6f: Deleting consensus metadata
I20260812 04:00:12.589672 15242 catalog_manager.cc:4841] TS b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077): tablet a74f5949113644d29371a59d2933be84 (table test-table [id=6810587146c140f6a79772ef03a3ce9e]) successfully deleted
I20260812 04:00:12.594446 15816 tablet_replica.cc:330] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7: stopping tablet replica
I20260812 04:00:12.594624 15816 raft_consensus.cc:2229] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:00:12.594823 15816 raft_consensus.cc:2258] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 04:00:12.599661 15239 connection.cc:535] client connection to 127.31.43.193:36067 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 04:00:12.599821 15239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 04:00:12.599987 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 15478
I20260812 04:00:12.600409 15816 ts_tablet_manager.cc:1905] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:00:12.600453 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.600325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:36067 (local address 127.0.0.1:35682)
0812 04:00:12.600382 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:12.600414 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 04:00:12.600598 15239 catalog_manager.cc:4584] TS 1c8af0f13ad1411c9c3bc1553a4eb1cd (127.31.43.193:36067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a74f5949113644d29371a59d2933be84: Network error: Client connection negotiation failed: client connection to 127.31.43.193:36067: connect: Connection refused (error 111)
I20260812 04:00:12.603665 15816 ts_tablet_manager.cc:1918] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:00:12.603765 15816 log.cc:1196] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/a74f5949113644d29371a59d2933be84
I20260812 04:00:12.604069 15816 ts_tablet_manager.cc:1939] T a74f5949113644d29371a59d2933be84 P 637df5a1285446a29ffcc32ea70051a7: Deleting consensus metadata
I20260812 04:00:12.604631 15242 catalog_manager.cc:4841] TS 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713): tablet a74f5949113644d29371a59d2933be84 (table test-table [id=6810587146c140f6a79772ef03a3ce9e]) successfully deleted
I20260812 04:00:12.608358 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 15629
I20260812 04:00:12.617849 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 15220
I20260812 04:00:12.627746 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40475
--webserver_interface=127.31.43.254
--webserver_port=44615
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:41269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:40475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:12.717036 15819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:12.717269 15819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:12.717296 15819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:12.718565 15819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:00:12.718618 15819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:12.718637 15819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:12.718650 15819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:00:12.718669 15819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:00:12.720327 15819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:41269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:40475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=44615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:12.720615 15819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:12.720870 15819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:12.733945 15828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:12.733997 15825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:12.734262 15819 server_base.cc:1034] running on GCE node
W20260812 04:00:12.733945 15826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:12.734658 15819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:12.735088 15819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:12.736285 15819 hybrid_clock.cc:648] HybridClock initialized: now 1786507212736258 us; error 55 us; skew 500 ppm
Aug 12 04:00:12 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507212, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:12.739459 15819 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:00:12.740000 15819 webserver.cc:460] Webserver started at http://127.31.43.254:44615/ using document root <none> and password file <none>
I20260812 04:00:12.740221 15819 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:12.740278 15819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:12.741776 15819 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 04:00:12.742583 15836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:12.742852 15819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 04:00:12.742939 15819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "abd36733c3fc40dab1f4ac8ee45cd026"
format_stamp: "Formatted at 2026-08-12 04:00:11 on dist-test-slave-8tdl"
server_key: "f7313cbf4a397f9069ad7330ff373fa9"
server_key_iv: "b4999656c794834780de18a622640ae0"
server_key_version: "encryptionkey@0"
I20260812 04:00:12.743361 15819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:12.756592 15819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:12.758754 15819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:12.758881 15819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:12.763327 15819 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:40475
I20260812 04:00:12.763366 15897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:40475 every 8 connection(s)
I20260812 04:00:12.763799 15819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:00:12.764144 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 15819
I20260812 04:00:12.764526 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:36067
--local_ip_for_outbound_sockets=127.31.43.193
--tserver_master_addrs=127.31.43.254:40475
--webserver_port=37235
--webserver_interface=127.31.43.193
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:41269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:00:12.766458 15898 sys_catalog.cc:263] Verifying existing consensus state
I20260812 04:00:12.767323 15898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Bootstrap starting.
I20260812 04:00:12.777208 15898 log.cc:824] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:12.779625 15898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 04:00:12.779970 15898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Bootstrap complete.
I20260812 04:00:12.782497 15898 raft_consensus.cc:348] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } }
I20260812 04:00:12.782845 15898 raft_consensus.cc:729] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd36733c3fc40dab1f4ac8ee45cd026, State: Initialized, Role: FOLLOWER
I20260812 04:00:12.783035 15898 consensus_queue.cc:260] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } }
I20260812 04:00:12.783098 15898 raft_consensus.cc:388] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:00:12.783167 15898 raft_consensus.cc:482] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:00:12.783224 15898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 1 FOLLOWER]: Advancing to term 2
I20260812 04:00:12.784376 15898 raft_consensus.cc:504] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } }
I20260812 04:00:12.784474 15898 leader_election.cc:302] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abd36733c3fc40dab1f4ac8ee45cd026; no voters:
I20260812 04:00:12.784749 15898 leader_election.cc:288] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 04:00:12.784789 15902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 04:00:12.784905 15902 raft_consensus.cc:686] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 2 LEADER]: Becoming Leader. State: Replica: abd36733c3fc40dab1f4ac8ee45cd026, State: Running, Role: LEADER
I20260812 04:00:12.785038 15898 sys_catalog.cc:564] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:00:12.785006 15902 consensus_queue.cc:237] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } }
I20260812 04:00:12.785306 15902 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abd36733c3fc40dab1f4ac8ee45cd026. Latest consensus state: current_term: 2 leader_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } } }
I20260812 04:00:12.785401 15902 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:12.785631 15905 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } } }
I20260812 04:00:12.785686 15907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:00:12.785749 15905 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:12.786623 15907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:00:12.786791 15907 catalog_manager.cc:1259] Loaded cluster ID: 7d5a162f7196494a9ce5dde02431a856
I20260812 04:00:12.786836 15907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:00:12.787161 15907 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 04:00:12.790174 15907 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 04:00:12.795751 15907 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:00:12.796022 15907 catalog_manager.cc:5879] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Loaded TSK: 0
I20260812 04:00:12.796252 15907 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 04:00:12.863196 15900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:12.863425 15900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:12.863452 15900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:12.864730 15900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:12.864784 15900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:12.864814 15900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:00:12.866422 15900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:41269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:36067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=37235
--tserver_master_addrs=127.31.43.254:40475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:12.866690 15900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:12.866982 15900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:12.873641 15931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:12.873710 15900 server_base.cc:1034] running on GCE node
W20260812 04:00:12.873631 15928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:12.873653 15929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:12.874238 15900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:12.874547 15900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:12.875748 15900 hybrid_clock.cc:648] HybridClock initialized: now 1786507212875711 us; error 52 us; skew 500 ppm
Aug 12 04:00:12 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507212, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:12.878898 15900 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 04:00:12.879479 15900 webserver.cc:460] Webserver started at http://127.31.43.193:37235/ using document root <none> and password file <none>
I20260812 04:00:12.879768 15900 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:12.879863 15900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:12.881518 15900 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:00:12.882391 15939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:12.882599 15900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:12.882728 15900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "c4fed6b5c0a43a82c7d0d91d62165a39"
server_key_iv: "975d105db9bc5886788c448d070e7964"
server_key_version: "encryptionkey@0"
I20260812 04:00:12.883124 15900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:12.908807 15900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:12.911331 15900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:12.911504 15900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:12.912091 15900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:12.912751 15947 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 04:00:12.914386 15900 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 04:00:12.914448 15900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 04:00:12.914475 15900 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 04:00:12.915298 15900 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 04:00:12.915349 15900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 04:00:12.915396 15947 tablet_bootstrap.cc:492] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Bootstrap starting.
I20260812 04:00:12.923916 15900 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:36067
I20260812 04:00:12.924109 16068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:36067 every 8 connection(s)
I20260812 04:00:12.924494 15900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 04:00:12.929006 15947 log.cc:824] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:12.931826 15947 tablet_bootstrap.cc:492] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 04:00:12.931998 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 15900
I20260812 04:00:12.932301 15947 tablet_bootstrap.cc:492] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Bootstrap complete.
I20260812 04:00:12.932318 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:41077
--local_ip_for_outbound_sockets=127.31.43.194
--tserver_master_addrs=127.31.43.254:40475
--webserver_port=44717
--webserver_interface=127.31.43.194
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:41269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:00:12.933732 15947 ts_tablet_manager.cc:1397] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.010s
I20260812 04:00:12.934995 15947 raft_consensus.cc:348] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } } peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } }
I20260812 04:00:12.935474 15947 raft_consensus.cc:729] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c8af0f13ad1411c9c3bc1553a4eb1cd, State: Initialized, Role: FOLLOWER
I20260812 04:00:12.935636 15947 consensus_queue.cc:260] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } } peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } }
Aug 12 04:00:12 dist-test-slave-8tdl krb5kdc[15205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507212, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:12.935954 15947 ts_tablet_manager.cc:1428] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260812 04:00:12.941196 16071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.925325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:43113)
0812 04:00:12.925672 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:12.925721 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:12.926352 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:12.926913 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:12.926917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:12.927295 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:12.927556 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.927563 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.929359 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.929360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.930441 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.930445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.931197 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.931198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.931349 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.934243 (+ 2894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:12.934256 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:12.936385 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:12.939178 (+ 2793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:12.939186 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:12.939188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:12.939469 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:12.939828 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:12.939828 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:12.939829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:12.939996 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:12.940579 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:12.940586 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:12.940711 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 04:00:12.940816 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 04:00:12.941030 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":106,"threads_started":1}
I20260812 04:00:12.941334 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.925433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:43113 (local address 127.31.43.254:40475)
0812 04:00:12.925661 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:12.925672 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:12.925865 (+ 193us) server_negotiation.cc:407] Connection header received
0812 04:00:12.926596 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:12.926604 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:12.926670 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:12.926870 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:12.927708 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.929240 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.930552 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.931042 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.931188 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.936541 (+ 5353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:12.936546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:12.936547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:12.936561 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:12.939054 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:12.939573 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:12.939577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:12.939578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:12.939750 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:12.940076 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:12.940077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:12.940077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:12.940471 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:12.940562 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:12.940917 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 04:00:12.941088 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":108,"threads_started":1}
I20260812 04:00:12.942056 16069 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40475
I20260812 04:00:12.942183 16069 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:12.942423 16069 heartbeater.cc:511] Master 127.31.43.254:40475 requested a full tablet report, sending...
I20260812 04:00:12.943419 15854 ts_manager.cc:194] Registered new tserver with Master: 1c8af0f13ad1411c9c3bc1553a4eb1cd (127.31.43.193:36067)
I20260812 04:00:12.944650 15854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:43113
I20260812 04:00:12.945819 16069 heartbeater.cc:503] Master 127.31.43.254:40475 was elected leader, sending a full tablet report...
Aug 12 04:00:12 dist-test-slave-8tdl krb5kdc[15205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507212, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.193@KRBTEST.COM
I20260812 04:00:12.953229 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.944191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35694 (local address 127.31.43.193:36067)
0812 04:00:12.944845 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:12.944850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:12.944862 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:00:12.944898 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:12.944900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:12.944918 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:12.944996 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:12.945464 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.946349 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.947164 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.947489 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.947575 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.949958 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:12.949970 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:12.949970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:12.949980 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:12.951748 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:12.952251 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:12.952254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:12.952254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:12.952315 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:12.952591 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:12.952593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:12.952593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:12.952760 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:12.952840 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:12.953065 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 04:00:12.953157 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":99,"threads_started":1}
I20260812 04:00:12.953280 16076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:12.944076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:36067 (local address 127.0.0.1:35694)
0812 04:00:12.944335 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:12.944368 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:12.944434 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:12.945000 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:12.945004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:12.945286 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:12.945369 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.945372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.946416 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.946417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.947077 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:12.947080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:12.947575 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:12.947576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:12.947660 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:12.948215 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:12.948221 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:12.949846 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:12.951846 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:12.951854 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:12.951856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:12.952153 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:12.952395 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:12.952396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:12.952396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:12.952491 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:12.952846 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:12.952856 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:12.952921 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 04:00:12.953019 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 04:00:12.953154 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":109,"threads_started":1}
I20260812 04:00:12.953815 15991 tablet_service.cc:1514] Processing DeleteTablet for tablet a74f5949113644d29371a59d2933be84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:12 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:35694
I20260812 04:00:12.954535 16078 tablet_replica.cc:330] stopping tablet replica
I20260812 04:00:12.954671 16078 raft_consensus.cc:2229] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:00:12.954782 16078 raft_consensus.cc:2258] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:00:12.955168 16078 ts_tablet_manager.cc:1905] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:00:12.957228 16078 ts_tablet_manager.cc:1918] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:00:12.957326 16078 log.cc:1196] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal/wals/a74f5949113644d29371a59d2933be84
I20260812 04:00:12.957584 16078 ts_tablet_manager.cc:1939] T a74f5949113644d29371a59d2933be84 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Deleting consensus metadata
I20260812 04:00:12.958146 15837 catalog_manager.cc:4841] TS 1c8af0f13ad1411c9c3bc1553a4eb1cd (127.31.43.193:36067): tablet a74f5949113644d29371a59d2933be84 (table test-table [id=6810587146c140f6a79772ef03a3ce9e]) successfully deleted
W20260812 04:00:13.025708 16074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:13.025931 16074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:13.025956 16074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:13.027199 16074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:13.027261 16074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:13.027294 16074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:00:13.028810 16074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:41269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:41077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=44717
--tserver_master_addrs=127.31.43.254:40475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:13.029126 16074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:13.029387 16074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:13.034912 16085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:13.034942 16087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:13.035123 16074 server_base.cc:1034] running on GCE node
W20260812 04:00:13.034976 16084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:13.035419 16074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:13.035701 16074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:13.036877 16074 hybrid_clock.cc:648] HybridClock initialized: now 1786507213036877 us; error 81 us; skew 500 ppm
Aug 12 04:00:13 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507213, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:13.039553 16074 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 04:00:13.040066 16074 webserver.cc:460] Webserver started at http://127.31.43.194:44717/ using document root <none> and password file <none>
I20260812 04:00:13.040313 16074 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:13.040407 16074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:13.041855 16074 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:00:13.042580 16094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:13.042783 16074 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:13.042899 16074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "b528ed2b2162480a8bbcf21edfc18f6f"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "fc877be0679493e555011a3854dad37f"
server_key_iv: "3f8d72854995e68c1ae72bb4591452a7"
server_key_version: "encryptionkey@0"
I20260812 04:00:13.043292 16074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:13.057072 16074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:13.059262 16074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:13.059428 16074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:13.059705 16074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:13.060039 16074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:13.060099 16074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:13.060153 16074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:13.060185 16074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:13.066298 16074 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:41077
I20260812 04:00:13.066349 16219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:41077 every 8 connection(s)
I20260812 04:00:13.066874 16074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:00:13.070585 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 16074
I20260812 04:00:13.070915 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:46713
--local_ip_for_outbound_sockets=127.31.43.195
--tserver_master_addrs=127.31.43.254:40475
--webserver_port=44669
--webserver_interface=127.31.43.195
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:41269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 04:00:13 dist-test-slave-8tdl krb5kdc[15205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507213, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:13.078030 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.067770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:50663 (local address 127.31.43.254:40475)
0812 04:00:13.067819 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:13.067821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:13.068069 (+ 248us) server_negotiation.cc:407] Connection header received
0812 04:00:13.068615 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:13.068617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:13.068631 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:13.068713 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:13.069727 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.070326 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.071677 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.071898 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.071987 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.074585 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:13.074588 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:13.074589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:13.074600 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:13.076241 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:13.076831 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:13.076833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:13.076833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:13.076893 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:13.077235 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:13.077236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:13.077237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:13.077372 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:13.077431 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:13.077811 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 04:00:13.077895 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 04:00:13.078290 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.067747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:50663)
0812 04:00:13.067914 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:13.067962 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:13.068481 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:13.068767 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:13.068770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:13.069404 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:13.069618 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:13.069623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.070450 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.070451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:13.071567 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:13.071570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.071993 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.071994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:13.072125 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.072682 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:13.072691 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:13.074452 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:13.076331 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:13.076341 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:13.076344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:13.076730 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:13.076965 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:13.076966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:13.076966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:13.077151 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:13.077452 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:13.077460 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:13.077657 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 04:00:13.077792 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 04:00:13.078001 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260812 04:00:13.078967 16220 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40475
I20260812 04:00:13.079104 16220 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:13.079305 16220 heartbeater.cc:511] Master 127.31.43.254:40475 requested a full tablet report, sending...
I20260812 04:00:13.079794 15854 ts_manager.cc:194] Registered new tserver with Master: b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077)
I20260812 04:00:13.080207 15854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:50663
W20260812 04:00:13.163854 16224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:13.164115 16224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:13.164156 16224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:13.165373 16224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:13.165449 16224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:13.165498 16224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:00:13.167601 16224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:41269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:46713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=44669
--tserver_master_addrs=127.31.43.254:40475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:13.168001 16224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:13.168275 16224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:13.174237 16229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:13.174242 16232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:13.174247 16230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:13.174733 16224 server_base.cc:1034] running on GCE node
I20260812 04:00:13.175007 16224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:13.175280 16224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:13.176448 16224 hybrid_clock.cc:648] HybridClock initialized: now 1786507213176415 us; error 51 us; skew 500 ppm
Aug 12 04:00:13 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507213, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:13.179203 16224 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 04:00:13.179709 16224 webserver.cc:460] Webserver started at http://127.31.43.195:44669/ using document root <none> and password file <none>
I20260812 04:00:13.179942 16224 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:13.180022 16224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:13.181417 16224 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 04:00:13.182044 16240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:13.182236 16224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:13.182343 16224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "637df5a1285446a29ffcc32ea70051a7"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "7d3f7f5340b93984e8bd19341b80b5b0"
server_key_iv: "d591e86cc16a775eeece604ee8f04f19"
server_key_version: "encryptionkey@0"
I20260812 04:00:13.182796 16224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:13.190558 16224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:13.192960 16224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:13.193133 16224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:13.193379 16224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:13.193713 16224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:13.193768 16224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:13.193811 16224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:13.193840 16224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:13.199785 16224 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:46713
I20260812 04:00:13.199867 16367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:46713 every 8 connection(s)
I20260812 04:00:13.200223 16224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
Aug 12 04:00:13 dist-test-slave-8tdl krb5kdc[15205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507213, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:13.208349 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 16224
I20260812 04:00:13.209931 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.201066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:40951 (local address 127.31.43.254:40475)
0812 04:00:13.201137 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:13.201139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:13.201317 (+ 178us) server_negotiation.cc:407] Connection header received
0812 04:00:13.201818 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:13.201819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:13.201829 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:13.201893 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:13.202933 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.203431 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.204390 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.204575 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.204632 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.206882 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:13.206884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:13.206884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:13.206892 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:13.208265 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:13.208733 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:13.208734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:13.208735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:13.208801 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:13.209166 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:13.209167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:13.209168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:13.209295 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:13.209351 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:13.209744 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 04:00:13.209843 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 04:00:13.210178 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.201022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:40951)
0812 04:00:13.201185 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:13.201229 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:13.201723 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:13.201953 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:13.201956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:13.202559 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:13.202839 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:13.202845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.203515 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.203515 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:13.204310 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:13.204312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.204654 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.204655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:13.204752 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.205306 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:13.205315 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:13.206792 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:13.208345 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:13.208351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:13.208354 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:13.208645 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:13.208908 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:13.208909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:13.208909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:13.209069 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:13.209366 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:13.209374 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:13.209557 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 04:00:13.209723 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 04:00:13.209909 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260812 04:00:13.210752 16368 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40475
I20260812 04:00:13.210868 16368 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:13.211067 16368 heartbeater.cc:511] Master 127.31.43.254:40475 requested a full tablet report, sending...
I20260812 04:00:13.211519 15854 ts_manager.cc:194] Registered new tserver with Master: 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713)
I20260812 04:00:13.211884 15854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:40951
I20260812 04:00:13.215790 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.208776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42518 (local address 127.31.43.254:40475)
0812 04:00:13.208892 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:13.208894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:13.208988 (+ 94us) server_negotiation.cc:407] Connection header received
0812 04:00:13.209047 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:13.209048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:13.209060 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:13.209120 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:13.209842 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.210853 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.211731 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.211951 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.212022 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.213289 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:13.213291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:13.213291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:13.213298 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:13.214497 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:13.214956 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:13.214958 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:13.214958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:13.215018 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:13.215225 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:13.215226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:13.215226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:13.215341 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:13.215526 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:13.215590 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 04:00:13.215675 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":47,"threads_started":1}
I20260812 04:00:13.216548 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 04:00:13.219230 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.216900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42534 (local address 127.31.43.254:40475)
0812 04:00:13.216949 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:13.216950 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:13.217042 (+ 92us) server_negotiation.cc:407] Connection header received
0812 04:00:13.217132 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:13.217133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:13.217161 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:13.217222 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:00:13.217321 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.217851 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.218539 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.218777 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.218831 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.218965 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:00:13.219058 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:00:13.219124 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:13.219157 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 04:00:13.219175 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 04:00:13.219501 15854 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 04:00:13.221138 15854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 04:00:13.223389 15991 tablet_service.cc:1467] Processing CreateTablet for tablet 312b3cd97a3c462280d7c169eb4e5f70 (DEFAULT_TABLE table=test-table [id=2d72741cf6264dc0abe651db5e56b7ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:13.223608 15991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 312b3cd97a3c462280d7c169eb4e5f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:13.225051 15947 tablet_bootstrap.cc:492] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Bootstrap starting.
I20260812 04:00:13.225387 15947 tablet_bootstrap.cc:654] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:13.226065 15947 tablet_bootstrap.cc:492] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: No bootstrap required, opened a new log
I20260812 04:00:13.226125 15947 ts_tablet_manager.cc:1397] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:13.226473 15947 raft_consensus.cc:348] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:13.226567 15947 raft_consensus.cc:374] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:13.226637 15947 raft_consensus.cc:729] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c8af0f13ad1411c9c3bc1553a4eb1cd, State: Initialized, Role: FOLLOWER
I20260812 04:00:13.226704 15947 consensus_queue.cc:260] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:13.226812 15947 ts_tablet_manager.cc:1428] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 04:00:13.227728 16076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.223091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:46713 (local address 127.0.0.1:35948)
0812 04:00:13.223154 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:13.223165 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:13.223204 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:13.223761 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:13.223762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:13.223764 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:13.223811 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:13.223813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.225204 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.225205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:13.226380 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:13.226383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.227455 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.227456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:13.227528 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.227563 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 04:00:13.227620 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 04:00:13.227667 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":28}
I20260812 04:00:13.227785 16376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.223265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35948 (local address 127.31.43.195:46713)
0812 04:00:13.223556 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:13.223561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:13.223576 (+ 15us) server_negotiation.cc:407] Connection header received
0812 04:00:13.223607 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:13.223609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:13.223621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:13.223887 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:13.223958 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.225068 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.226471 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.227299 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.227368 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.227421 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:13.227650 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 04:00:13.227701 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":104,"threads_started":1}
I20260812 04:00:13.228514 16294 tablet_service.cc:1467] Processing CreateTablet for tablet 312b3cd97a3c462280d7c169eb4e5f70 (DEFAULT_TABLE table=test-table [id=2d72741cf6264dc0abe651db5e56b7ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:13.228677 16375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.223212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:41077 (local address 127.0.0.1:40428)
0812 04:00:13.223391 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:13.223403 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:13.223451 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:13.223894 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:13.223895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:13.223898 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:13.223954 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:13.223955 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.225223 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.225223 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:13.227525 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:13.227527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.228421 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.228423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:13.228489 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.228494 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 04:00:13.228565 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 04:00:13.228606 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":42,"thread_start_us":134,"threads_started":1}
I20260812 04:00:13.228689 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.223505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40428 (local address 127.31.43.194:41077)
0812 04:00:13.223726 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:13.223730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:13.223739 (+ 9us) server_negotiation.cc:407] Connection header received
0812 04:00:13.223777 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:13.223779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:13.223795 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:13.223877 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:13.224034 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.225117 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.227639 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.228337 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.228397 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.228430 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:13.228577 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 04:00:13.228626 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":128,"threads_started":1}
I20260812 04:00:13.228941 16294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 312b3cd97a3c462280d7c169eb4e5f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:13.229619 16145 tablet_service.cc:1467] Processing CreateTablet for tablet 312b3cd97a3c462280d7c169eb4e5f70 (DEFAULT_TABLE table=test-table [id=2d72741cf6264dc0abe651db5e56b7ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:13.229947 16145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 312b3cd97a3c462280d7c169eb4e5f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:13.231983 16381 tablet_bootstrap.cc:492] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7: Bootstrap starting.
I20260812 04:00:13.232625 16381 tablet_bootstrap.cc:654] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:13.232751 16382 tablet_bootstrap.cc:492] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f: Bootstrap starting.
I20260812 04:00:13.233239 16381 log.cc:824] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:13.233480 16382 tablet_bootstrap.cc:654] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:13.234068 16382 log.cc:824] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:13.234134 16381 tablet_bootstrap.cc:492] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7: No bootstrap required, opened a new log
I20260812 04:00:13.234193 16381 ts_tablet_manager.cc:1397] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 04:00:13.234915 16382 tablet_bootstrap.cc:492] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f: No bootstrap required, opened a new log
I20260812 04:00:13.234994 16382 ts_tablet_manager.cc:1397] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 04:00:13.236250 16381 raft_consensus.cc:348] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:13.236436 16381 raft_consensus.cc:374] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:13.236485 16381 raft_consensus.cc:729] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 637df5a1285446a29ffcc32ea70051a7, State: Initialized, Role: FOLLOWER
I20260812 04:00:13.236610 16381 consensus_queue.cc:260] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:13.236903 16381 ts_tablet_manager.cc:1428] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 04:00:13.236955 16368 heartbeater.cc:503] Master 127.31.43.254:40475 was elected leader, sending a full tablet report...
I20260812 04:00:13.236979 16382 raft_consensus.cc:348] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:13.237121 16382 raft_consensus.cc:374] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:13.237149 16382 raft_consensus.cc:729] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b528ed2b2162480a8bbcf21edfc18f6f, State: Initialized, Role: FOLLOWER
I20260812 04:00:13.237260 16382 consensus_queue.cc:260] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:13.237552 16220 heartbeater.cc:503] Master 127.31.43.254:40475 was elected leader, sending a full tablet report...
I20260812 04:00:13.237789 16382 ts_tablet_manager.cc:1428] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260812 04:00:13.317624 16222 tablet.cc:2369] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 04:00:13.425570 16070 tablet.cc:2369] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 04:00:13.450873 16369 tablet.cc:2369] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:00:13.662786 16385 raft_consensus.cc:482] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:00:13.663007 16385 raft_consensus.cc:504] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:13.663478 16385 leader_election.cc:288] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077), 1c8af0f13ad1411c9c3bc1553a4eb1cd (127.31.43.193:36067)
I20260812 04:00:13.667763 16387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.663802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:58633 (local address 127.31.43.193:36067)
0812 04:00:13.663965 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:13.663967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:13.663976 (+ 9us) server_negotiation.cc:407] Connection header received
0812 04:00:13.664004 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:13.664005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:13.664087 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:13.664169 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:13.664398 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.665202 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.666851 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.667324 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.667423 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.667501 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:13.667624 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 04:00:13.667675 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
I20260812 04:00:13.667762 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.663701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:36067 (local address 127.31.43.195:58633)
0812 04:00:13.663774 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:13.663790 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:13.663863 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:13.664189 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:13.664190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:13.664191 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:13.664311 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:13.664313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.665305 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.665307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:13.666736 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:13.666742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.667428 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.667430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:13.667547 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.667551 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:00:13.667597 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 04:00:13.667666 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":83}
I20260812 04:00:13.667802 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.663842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:59407 (local address 127.31.43.194:41077)
0812 04:00:13.663890 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:13.663892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:13.664053 (+ 161us) server_negotiation.cc:407] Connection header received
0812 04:00:13.664087 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:13.664087 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:13.664097 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:13.664168 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:13.664397 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.665163 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.666939 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.667322 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.667440 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.667451 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:13.667657 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 04:00:13.667723 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 04:00:13.667778 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.663709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:41077 (local address 127.31.43.195:59407)
0812 04:00:13.663961 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:13.663972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:13.664037 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:13.664189 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:13.664190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:13.664191 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:13.664311 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:13.664313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.665306 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.665307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:13.666820 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:13.666825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.667444 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.667445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:13.667552 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.667556 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:00:13.667633 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 04:00:13.667676 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":225,"thread_start_us":92,"threads_started":1}
I20260812 04:00:13.668205 16015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "312b3cd97a3c462280d7c169eb4e5f70" candidate_uuid: "637df5a1285446a29ffcc32ea70051a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" is_pre_election: true
I20260812 04:00:13.668280 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "312b3cd97a3c462280d7c169eb4e5f70" candidate_uuid: "637df5a1285446a29ffcc32ea70051a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" is_pre_election: true
I20260812 04:00:13.668386 16015 raft_consensus.cc:2454] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 637df5a1285446a29ffcc32ea70051a7 in term 0.
I20260812 04:00:13.668406 16169 raft_consensus.cc:2454] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 637df5a1285446a29ffcc32ea70051a7 in term 0.
I20260812 04:00:13.668639 16241 leader_election.cc:302] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c8af0f13ad1411c9c3bc1553a4eb1cd, 637df5a1285446a29ffcc32ea70051a7; no voters:
I20260812 04:00:13.668810 16385 raft_consensus.cc:2781] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 04:00:13.668869 16385 raft_consensus.cc:482] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 04:00:13.668884 16385 raft_consensus.cc:3037] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:13.669647 16385 raft_consensus.cc:504] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:13.669797 16385 leader_election.cc:288] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [CANDIDATE]: Term 1 election: Requested vote from peers b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077), 1c8af0f13ad1411c9c3bc1553a4eb1cd (127.31.43.193:36067)
I20260812 04:00:13.669965 16015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "312b3cd97a3c462280d7c169eb4e5f70" candidate_uuid: "637df5a1285446a29ffcc32ea70051a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd"
I20260812 04:00:13.670048 16015 raft_consensus.cc:3037] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:13.670204 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "312b3cd97a3c462280d7c169eb4e5f70" candidate_uuid: "637df5a1285446a29ffcc32ea70051a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b528ed2b2162480a8bbcf21edfc18f6f"
I20260812 04:00:13.670323 16169 raft_consensus.cc:3037] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:13.670897 16015 raft_consensus.cc:2454] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 637df5a1285446a29ffcc32ea70051a7 in term 1.
I20260812 04:00:13.671075 16241 leader_election.cc:302] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c8af0f13ad1411c9c3bc1553a4eb1cd, 637df5a1285446a29ffcc32ea70051a7; no voters:
I20260812 04:00:13.671186 16385 raft_consensus.cc:2781] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:00:13.671371 16385 raft_consensus.cc:686] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 1 LEADER]: Becoming Leader. State: Replica: 637df5a1285446a29ffcc32ea70051a7, State: Running, Role: LEADER
I20260812 04:00:13.671375 16169 raft_consensus.cc:2454] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 637df5a1285446a29ffcc32ea70051a7 in term 1.
I20260812 04:00:13.671480 16385 consensus_queue.cc:237] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:13.672537 15853 catalog_manager.cc:5495] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195). New cstate: current_term: 1 leader_uuid: "637df5a1285446a29ffcc32ea70051a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } health_report { overall_health: UNKNOWN } } }
I20260812 04:00:13.782743 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.780077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.31.43.195:46713)
0812 04:00:13.780249 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:13.780251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:13.780259 (+ 8us) server_negotiation.cc:407] Connection header received
0812 04:00:13.780309 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:13.780309 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:13.780336 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:13.780428 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:00:13.780601 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.781101 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.781876 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:13.782108 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:13.782187 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:13.782352 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:00:13.782470 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:00:13.782571 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:13.782596 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 04:00:13.782619 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
I20260812 04:00:13.784761 16169 raft_consensus.cc:1261] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f [term 1 FOLLOWER]: Refusing update from remote peer 637df5a1285446a29ffcc32ea70051a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:00:13.784761 16015 raft_consensus.cc:1261] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Refusing update from remote peer 637df5a1285446a29ffcc32ea70051a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:00:13.785153 16390 consensus_queue.cc:1035] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:00:13.785343 16385 consensus_queue.cc:1035] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:00:13.788769 16400 mvcc.cc:205] Tried to move back new op lower bound from 7317533547659235328 to 7317533547199107072. Current Snapshot: MvccSnapshot[applied={T|T < 7317533547659235328}]
I20260812 04:00:13.790308 15853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:00:13.790433 15853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:00:13.791335 15853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Sending DeleteTablet for 3 replicas of tablet 312b3cd97a3c462280d7c169eb4e5f70
I20260812 04:00:13.791776 16294 tablet_service.cc:1514] Processing DeleteTablet for tablet 312b3cd97a3c462280d7c169eb4e5f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:13 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:35948
I20260812 04:00:13.792047 16145 tablet_service.cc:1514] Processing DeleteTablet for tablet 312b3cd97a3c462280d7c169eb4e5f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:13 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:40428
I20260812 04:00:13.792603 15991 tablet_service.cc:1514] Processing DeleteTablet for tablet 312b3cd97a3c462280d7c169eb4e5f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:13 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:35694
I20260812 04:00:13.792996 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 15900
I20260812 04:00:13.794984 16408 tablet_replica.cc:330] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7: stopping tablet replica
I20260812 04:00:13.795153 16408 raft_consensus.cc:2229] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 04:00:13.795306 16408 raft_consensus.cc:2258] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:00:13.796484 16408 ts_tablet_manager.cc:1905] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:00:13.798802 16408 ts_tablet_manager.cc:1918] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:00:13.798897 16408 log.cc:1196] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/312b3cd97a3c462280d7c169eb4e5f70
I20260812 04:00:13.799201 16408 ts_tablet_manager.cc:1939] T 312b3cd97a3c462280d7c169eb4e5f70 P 637df5a1285446a29ffcc32ea70051a7: Deleting consensus metadata
I20260812 04:00:13.800800 16401 mvcc.cc:205] Tried to move back new op lower bound from 7317533547659235328 to 7317533547199107072. Current Snapshot: MvccSnapshot[applied={T|T < 7317533547659235328 or (T in {7317533547659235328})}]
I20260812 04:00:13.801530 15840 catalog_manager.cc:4841] TS 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713): tablet 312b3cd97a3c462280d7c169eb4e5f70 (table test-table [id=2d72741cf6264dc0abe651db5e56b7ee]) successfully deleted
I20260812 04:00:13.802893 16411 tablet_replica.cc:330] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f: stopping tablet replica
I20260812 04:00:13.803058 16411 raft_consensus.cc:2229] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:00:13.803224 16411 raft_consensus.cc:2258] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:00:13.803647 16411 ts_tablet_manager.cc:1905] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:00:13.805470 16411 ts_tablet_manager.cc:1918] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:00:13.805594 16411 log.cc:1196] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/312b3cd97a3c462280d7c169eb4e5f70
I20260812 04:00:13.805852 16411 ts_tablet_manager.cc:1939] T 312b3cd97a3c462280d7c169eb4e5f70 P b528ed2b2162480a8bbcf21edfc18f6f: Deleting consensus metadata
I20260812 04:00:13.806371 15840 catalog_manager.cc:4841] TS b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077): tablet 312b3cd97a3c462280d7c169eb4e5f70 (table test-table [id=2d72741cf6264dc0abe651db5e56b7ee]) successfully deleted
W20260812 04:00:13.807322 15837 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:36067 (error 108)
I20260812 04:00:13.807858 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 16074
I20260812 04:00:13.809237 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:13.807700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:36067 (local address 127.0.0.1:35700)
0812 04:00:13.809118 (+ 1418us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:13.809180 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1395,"thread_start_us":57,"threads_started":1}
W20260812 04:00:13.809372 15837 catalog_manager.cc:4584] TS 1c8af0f13ad1411c9c3bc1553a4eb1cd (127.31.43.193:36067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 312b3cd97a3c462280d7c169eb4e5f70: Network error: Client connection negotiation failed: client connection to 127.31.43.193:36067: connect: Connection refused (error 111)
I20260812 04:00:13.821642 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 16224
I20260812 04:00:13.833540 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 15819
I20260812 04:00:13.843860 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40475
--webserver_interface=127.31.43.254
--webserver_port=44615
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:41269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:40475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:13.931284 16415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:13.931516 16415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:13.931542 16415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:13.932812 16415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:00:13.932857 16415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:13.932871 16415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:13.932884 16415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:00:13.932921 16415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:00:13.934562 16415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:41269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:40475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=44615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:13.934881 16415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:13.935144 16415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:13.940537 16423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:13.940537 16420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:13.940564 16421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:13.940788 16415 server_base.cc:1034] running on GCE node
I20260812 04:00:13.941105 16415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:13.941440 16415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:13.942598 16415 hybrid_clock.cc:648] HybridClock initialized: now 1786507213942585 us; error 33 us; skew 500 ppm
Aug 12 04:00:13 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507213, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:13.945477 16415 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:00:13.945955 16415 webserver.cc:460] Webserver started at http://127.31.43.254:44615/ using document root <none> and password file <none>
I20260812 04:00:13.946236 16415 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:13.946286 16415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:13.947789 16415 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:13.948499 16432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:13.948768 16415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:13.948894 16415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "abd36733c3fc40dab1f4ac8ee45cd026"
format_stamp: "Formatted at 2026-08-12 04:00:11 on dist-test-slave-8tdl"
server_key: "f7313cbf4a397f9069ad7330ff373fa9"
server_key_iv: "b4999656c794834780de18a622640ae0"
server_key_version: "encryptionkey@0"
I20260812 04:00:13.949297 16415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:13.969197 16415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:13.971726 16415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:13.971915 16415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:13.976277 16415 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:40475
I20260812 04:00:13.976336 16491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:40475 every 8 connection(s)
I20260812 04:00:13.976742 16415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:00:13.978493 16492 sys_catalog.cc:263] Verifying existing consensus state
I20260812 04:00:13.979234 16492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Bootstrap starting.
I20260812 04:00:13.980218 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 16415
I20260812 04:00:13.980545 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:36067
--local_ip_for_outbound_sockets=127.31.43.193
--tserver_master_addrs=127.31.43.254:40475
--webserver_port=37235
--webserver_interface=127.31.43.193
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:41269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:00:13.989543 16492 log.cc:824] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:13.991998 16492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 04:00:13.992342 16492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026: Bootstrap complete.
I20260812 04:00:13.994810 16492 raft_consensus.cc:348] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } }
I20260812 04:00:13.995182 16492 raft_consensus.cc:729] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd36733c3fc40dab1f4ac8ee45cd026, State: Initialized, Role: FOLLOWER
I20260812 04:00:13.995380 16492 consensus_queue.cc:260] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } }
I20260812 04:00:13.995440 16492 raft_consensus.cc:388] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:00:13.995504 16492 raft_consensus.cc:482] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:00:13.995555 16492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 2 FOLLOWER]: Advancing to term 3
I20260812 04:00:13.996618 16492 raft_consensus.cc:504] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } }
I20260812 04:00:13.996711 16492 leader_election.cc:302] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abd36733c3fc40dab1f4ac8ee45cd026; no voters:
I20260812 04:00:13.996934 16492 leader_election.cc:288] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 04:00:13.996977 16497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 04:00:13.997102 16497 raft_consensus.cc:686] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [term 3 LEADER]: Becoming Leader. State: Replica: abd36733c3fc40dab1f4ac8ee45cd026, State: Running, Role: LEADER
I20260812 04:00:13.997238 16492 sys_catalog.cc:564] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:00:13.997200 16497 consensus_queue.cc:237] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } }
I20260812 04:00:13.997547 16497 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abd36733c3fc40dab1f4ac8ee45cd026. Latest consensus state: current_term: 3 leader_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } } }
I20260812 04:00:13.997613 16497 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:13.997814 16498 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd36733c3fc40dab1f4ac8ee45cd026" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40475 } } }
I20260812 04:00:13.997884 16498 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd36733c3fc40dab1f4ac8ee45cd026 [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:13.997838 16501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:00:13.998827 16501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:00:13.998934 16501 catalog_manager.cc:1259] Loaded cluster ID: 7d5a162f7196494a9ce5dde02431a856
I20260812 04:00:13.998968 16501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 04:00:13.999467 16501 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd86ee559ad google::LogMessage::Fail() at ??:0
@ 0x7fd86ee5888a google::LogMessage::SendToLog() at ??:0
@ 0x7fd86ee554de google::LogMessage::Flush() at ??:0
@ 0x7fd86ee56a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd876d760f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fd876d7f8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd870d230d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd870d18954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd871b7c6db start_thread at ??:0
@ 0x7fd86d95361f clone at ??:0
W20260812 04:00:14.083395 16495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:14.083632 16495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:14.083664 16495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:14.084843 16495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:14.084888 16495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:14.084918 16495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:00:14.086476 16495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:41269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:36067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=37235
--tserver_master_addrs=127.31.43.254:40475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:14.086791 16495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:14.087044 16495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:14.092710 16538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:14.092824 16535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:14.092744 16495 server_base.cc:1034] running on GCE node
W20260812 04:00:14.092710 16536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:14.093120 16495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:14.093432 16495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:14.094597 16495 hybrid_clock.cc:648] HybridClock initialized: now 1786507214094585 us; error 28 us; skew 500 ppm
Aug 12 04:00:14 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507214, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:14.097551 16495 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 04:00:14.098105 16495 webserver.cc:460] Webserver started at http://127.31.43.193:37235/ using document root <none> and password file <none>
I20260812 04:00:14.098410 16495 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:14.098491 16495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:14.100236 16495 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:14.101192 16547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:14.101420 16495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:14.101528 16495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "c4fed6b5c0a43a82c7d0d91d62165a39"
server_key_iv: "975d105db9bc5886788c448d070e7964"
server_key_version: "encryptionkey@0"
I20260812 04:00:14.101959 16495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:14.130738 16495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:14.133273 16495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:14.133457 16495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:14.133735 16495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:14.134224 16556 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 04:00:14.136525 16495 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 04:00:14.136620 16495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260812 04:00:14.136675 16495 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 04:00:14.137449 16495 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 04:00:14.137507 16495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 04:00:14.137594 16556 tablet_bootstrap.cc:492] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Bootstrap starting.
I20260812 04:00:14.145325 16495 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:36067
I20260812 04:00:14.145503 16673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:36067 every 8 connection(s)
I20260812 04:00:14.145802 16495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 04:00:14.147596 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 16495
I20260812 04:00:14.147919 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:41077
--local_ip_for_outbound_sockets=127.31.43.194
--tserver_master_addrs=127.31.43.254:40475
--webserver_port=44717
--webserver_interface=127.31.43.194
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:41269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:00:14.149521 16556 log.cc:824] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:14.152783 16556 tablet_bootstrap.cc:492] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 04:00:14.153163 16556 tablet_bootstrap.cc:492] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Bootstrap complete.
I20260812 04:00:14.154476 16556 ts_tablet_manager.cc:1397] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.012s
I20260812 04:00:14.156069 16556 raft_consensus.cc:348] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:14.156431 16556 raft_consensus.cc:729] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c8af0f13ad1411c9c3bc1553a4eb1cd, State: Initialized, Role: FOLLOWER
I20260812 04:00:14.156592 16556 consensus_queue.cc:260] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:14.156888 16556 ts_tablet_manager.cc:1428] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd: Time spent starting tablet: real 0.002s user 0.003s sys 0.003s
W20260812 04:00:14.201174 16677 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:00:14.146597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:46745)
0812 04:00:14.146823 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:14.146870 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:14.147444 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:14.200949 (+ 53505us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":134,"threads_started":1}
W20260812 04:00:14.201469 16674 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:40475 (0 consecutive failures): Network error: Failed to ping master at 127.31.43.254:40475: Client connection negotiation failed: client connection to 127.31.43.254:40475: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 04:00:14.201839 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:14.201711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:51865)
0812 04:00:14.201770 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:14.201815 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 04:00:14.202275 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:14.202183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:59961)
0812 04:00:14.202227 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:14.202255 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 04:00:14.202399 16674 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 04:00:14.237422 16678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:14.237648 16678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:14.237673 16678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:14.238917 16678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:14.238962 16678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:14.238991 16678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:00:14.240459 16678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:41269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:41077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=44717
--tserver_master_addrs=127.31.43.254:40475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:14.240650 16678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:14.240845 16678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:14.246363 16690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:14.246477 16678 server_base.cc:1034] running on GCE node
W20260812 04:00:14.246363 16687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:14.246368 16688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:14.246914 16678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:14.247186 16678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:14.248346 16678 hybrid_clock.cc:648] HybridClock initialized: now 1786507214248325 us; error 31 us; skew 500 ppm
Aug 12 04:00:14 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507214, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:14.251488 16678 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 04:00:14.252043 16678 webserver.cc:460] Webserver started at http://127.31.43.194:44717/ using document root <none> and password file <none>
I20260812 04:00:14.252280 16678 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:14.252328 16678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:14.253872 16678 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:14.254573 16698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:14.254814 16678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:14.255000 16678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "b528ed2b2162480a8bbcf21edfc18f6f"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "fc877be0679493e555011a3854dad37f"
server_key_iv: "3f8d72854995e68c1ae72bb4591452a7"
server_key_version: "encryptionkey@0"
I20260812 04:00:14.255383 16678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:14.272892 16678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:14.275447 16678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:14.275573 16678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:14.275897 16678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:14.276252 16678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:14.276309 16678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:14.276360 16678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:14.276396 16678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:14.282925 16678 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:41077
I20260812 04:00:14.283067 16826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:41077 every 8 connection(s)
I20260812 04:00:14.283397 16678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:00:14.284528 16830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:14.284086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:53909)
0812 04:00:14.284247 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:14.284326 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260812 04:00:14.284637 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 16678
I20260812 04:00:14.284909 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:46713
--local_ip_for_outbound_sockets=127.31.43.195
--tserver_master_addrs=127.31.43.254:40475
--webserver_port=44669
--webserver_interface=127.31.43.195
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:41269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:14.285104 16827 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:40475 (0 consecutive failures): Network error: Failed to ping master at 127.31.43.254:40475: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
I20260812 04:00:14.285395 16830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:14.285287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:33361)
0812 04:00:14.285343 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:14.285373 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 04:00:14.285727 16830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:14.285645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:51399)
0812 04:00:14.285689 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:14.285711 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 04:00:14.285851 16827 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 04:00:14.371809 16831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:14.372036 16831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:14.372061 16831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:14.373602 16831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:14.373668 16831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:14.373697 16831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:00:14.375480 16831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:41269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:46713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=44669
--tserver_master_addrs=127.31.43.254:40475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:14.375695 16831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:14.375908 16831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:14.381449 16838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:14.381492 16831 server_base.cc:1034] running on GCE node
W20260812 04:00:14.381441 16842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:14.381421 16837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:14.381877 16831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:14.382090 16831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:14.383266 16831 hybrid_clock.cc:648] HybridClock initialized: now 1786507214383251 us; error 44 us; skew 500 ppm
Aug 12 04:00:14 dist-test-slave-8tdl krb5kdc[15205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507214, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:14.386052 16831 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 04:00:14.386564 16831 webserver.cc:460] Webserver started at http://127.31.43.195:44669/ using document root <none> and password file <none>
I20260812 04:00:14.386845 16831 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:14.386902 16831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:14.388338 16831 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:14.389011 16850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:14.389245 16831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:14.389338 16831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "637df5a1285446a29ffcc32ea70051a7"
format_stamp: "Formatted at 2026-08-12 04:00:12 on dist-test-slave-8tdl"
server_key: "7d3f7f5340b93984e8bd19341b80b5b0"
server_key_iv: "d591e86cc16a775eeece604ee8f04f19"
server_key_version: "encryptionkey@0"
I20260812 04:00:14.389719 16831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:14.411680 16831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:14.414012 16831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:14.414151 16831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:14.414366 16831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:14.414781 16831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:14.414829 16831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:14.414855 16831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:14.414878 16831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:14.421106 16831 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:46713
I20260812 04:00:14.421209 16982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:46713 every 8 connection(s)
I20260812 04:00:14.421590 16831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 04:00:14.422747 16985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:14.422306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:60631)
0812 04:00:14.422462 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:14.422579 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
W20260812 04:00:14.423185 16983 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:40475 (0 consecutive failures): Network error: Failed to ping master at 127.31.43.254:40475: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
I20260812 04:00:14.423511 16985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:14.423416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:56573)
0812 04:00:14.423463 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:14.423491 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 04:00:14.423768 16985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:14.423717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:47145)
0812 04:00:14.423738 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:14.423757 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260812 04:00:14.423874 16983 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 04:00:14.431582 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 16831
I20260812 04:00:15.203121 16987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:15.202850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:48565)
0812 04:00:15.202998 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:15.203052 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260812 04:00:15.286443 16988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:15.286217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:42883)
0812 04:00:15.286342 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:15.286384 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260812 04:00:15.384461 16989 raft_consensus.cc:482] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:00:15.384598 16989 raft_consensus.cc:504] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:15.384991 16989 leader_election.cc:288] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713), b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077)
Aug 12 04:00:15 dist-test-slave-8tdl krb5kdc[15205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507214, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.195@KRBTEST.COM
Aug 12 04:00:15 dist-test-slave-8tdl krb5kdc[15205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507214, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.194@KRBTEST.COM
I20260812 04:00:15.397140 16991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:15.385526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:41865 (local address 127.31.43.195:46713)
0812 04:00:15.385715 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:15.385725 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:15.385745 (+ 20us) server_negotiation.cc:407] Connection header received
0812 04:00:15.386274 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:15.386277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:15.386337 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:15.386481 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:15.387361 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:15.388358 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:15.389413 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:15.389821 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:15.389949 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:15.392449 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:15.392456 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:15.392456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:15.392471 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:15.394735 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:15.395431 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:15.395435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:15.395435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:15.395571 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:15.395982 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:15.395984 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:15.395984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:15.396360 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:15.396466 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:15.396876 (+ 410us) server_negotiation.cc:299] Negotiation successful
0812 04:00:15.397030 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":72,"threads_started":1}
I20260812 04:00:15.397154 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:15.385666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:47067 (local address 127.31.43.194:41077)
0812 04:00:15.385827 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:15.385836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:15.385864 (+ 28us) server_negotiation.cc:407] Connection header received
0812 04:00:15.386290 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:15.386292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:15.386337 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:15.386492 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:15.387364 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:15.388225 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:15.389408 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:15.389821 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:15.389949 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:15.392875 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:15.392878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:15.392879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:15.392892 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:15.394753 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:15.395431 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:15.395435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:15.395435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:15.395571 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:15.395983 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:15.395984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:15.395984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:15.396360 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:15.396466 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:15.396878 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 04:00:15.397030 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":53,"threads_started":1}
I20260812 04:00:15.397177 16990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:15.385522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:41077 (local address 127.31.43.193:47067)
0812 04:00:15.385655 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:15.385666 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:15.385725 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:15.386586 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:15.386587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:15.387042 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:15.387234 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:15.387241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:15.388340 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:15.388341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:15.389313 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:15.389316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:15.389922 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:15.389923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:15.390047 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:15.390643 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:15.390651 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:15.392757 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:15.394857 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:15.394862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:15.394877 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:15.395334 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:15.395655 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:15.395655 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:15.395656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:15.395825 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:15.396473 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:15.396481 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:15.396643 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 04:00:15.396769 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 04:00:15.397042 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":165,"thread_start_us":73,"threads_started":1}
I20260812 04:00:15.397823 16987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:15.385382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:46713 (local address 127.31.43.193:41865)
0812 04:00:15.385466 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:15.385488 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:15.385573 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:15.386531 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:15.386534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:15.387018 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:15.387234 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:15.387241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:15.388484 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:15.388485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:15.389314 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:15.389316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:15.389925 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:15.389926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:15.390047 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:15.390725 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:15.390730 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:15.392321 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:15.394852 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:15.394862 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:15.394875 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:15.395334 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:15.395655 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:15.395655 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:15.395656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:15.395825 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:15.396470 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:15.396481 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:15.396644 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 04:00:15.396769 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 04:00:15.396983 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":30}
W20260812 04:00:15.398329 16552 leader_election.cc:341] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077): Not found: Tablet not found: 312b3cd97a3c462280d7c169eb4e5f70
W20260812 04:00:15.398530 16552 leader_election.cc:341] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713): Not found: Tablet not found: 312b3cd97a3c462280d7c169eb4e5f70
I20260812 04:00:15.398598 16552 leader_election.cc:302] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c8af0f13ad1411c9c3bc1553a4eb1cd; no voters: 637df5a1285446a29ffcc32ea70051a7, b528ed2b2162480a8bbcf21edfc18f6f
I20260812 04:00:15.398743 16989 raft_consensus.cc:2726] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 04:00:15.424459 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:15.424242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:48331)
0812 04:00:15.424373 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:15.424419 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260812 04:00:16.204068 16994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:16.203799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:57527)
0812 04:00:16.203954 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:16.204005 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
I20260812 04:00:16.287315 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:16.286997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:58611)
0812 04:00:16.287223 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:16.287268 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":69,"threads_started":1}
I20260812 04:00:16.425236 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:16.425000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:37821)
0812 04:00:16.425146 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:16.425189 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260812 04:00:17.204906 16997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:17.204672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:43559)
0812 04:00:17.204806 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:17.204855 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260812 04:00:17.288143 16998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:17.287891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:43379)
0812 04:00:17.288057 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:17.288094 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
I20260812 04:00:17.394912 16999 raft_consensus.cc:482] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:00:17.395016 16999 raft_consensus.cc:504] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:17.395241 16999 leader_election.cc:288] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713), b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077)
W20260812 04:00:17.395740 16552 leader_election.cc:341] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713): Not found: Tablet not found: 312b3cd97a3c462280d7c169eb4e5f70
W20260812 04:00:17.395915 16552 leader_election.cc:341] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077): Not found: Tablet not found: 312b3cd97a3c462280d7c169eb4e5f70
I20260812 04:00:17.395964 16552 leader_election.cc:302] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c8af0f13ad1411c9c3bc1553a4eb1cd; no voters: 637df5a1285446a29ffcc32ea70051a7, b528ed2b2162480a8bbcf21edfc18f6f
I20260812 04:00:17.396034 16999 raft_consensus.cc:2726] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 04:00:17.426041 17000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:17.425760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:46827)
0812 04:00:17.425911 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:17.425985 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260812 04:00:18.205684 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:18.205408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:46657)
0812 04:00:18.205589 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:18.205636 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":69,"threads_started":1}
I20260812 04:00:18.289065 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:18.288775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:35619)
0812 04:00:18.288937 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:18.288979 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":83,"threads_started":1}
I20260812 04:00:18.426884 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:18.426599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:36407)
0812 04:00:18.426793 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:18.426838 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":100,"threads_started":1}
I20260812 04:00:19.206537 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:19.206274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:42809)
0812 04:00:19.206435 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:19.206486 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260812 04:00:19.290148 17007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:19.289843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:39539)
0812 04:00:19.290026 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:19.290071 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
I20260812 04:00:19.427695 17008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:19.427440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:54505)
0812 04:00:19.427601 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:19.427645 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260812 04:00:19.757732 17009 raft_consensus.cc:482] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:00:19.757858 17009 raft_consensus.cc:504] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:19.758060 17009 leader_election.cc:288] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713), b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077)
W20260812 04:00:19.758533 16552 leader_election.cc:341] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713): Not found: Tablet not found: 312b3cd97a3c462280d7c169eb4e5f70
W20260812 04:00:19.758600 16552 leader_election.cc:341] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077): Not found: Tablet not found: 312b3cd97a3c462280d7c169eb4e5f70
I20260812 04:00:19.758654 16552 leader_election.cc:302] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c8af0f13ad1411c9c3bc1553a4eb1cd; no voters: 637df5a1285446a29ffcc32ea70051a7, b528ed2b2162480a8bbcf21edfc18f6f
I20260812 04:00:19.758759 17009 raft_consensus.cc:2726] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 04:00:20.207636 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:20.207262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:57327)
0812 04:00:20.207499 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:20.207558 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":100,"threads_started":1}
I20260812 04:00:20.291213 17012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:20.290891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:32985)
0812 04:00:20.291078 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:20.291152 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":111,"threads_started":1}
I20260812 04:00:20.428730 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:20.428337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:57763)
0812 04:00:20.428573 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:20.428654 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":142,"threads_started":1}
I20260812 04:00:21.208463 17015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:21.208188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:35427)
0812 04:00:21.208366 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:21.208411 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260812 04:00:21.292114 17016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:21.291844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:52623)
0812 04:00:21.292017 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:21.292062 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
I20260812 04:00:21.429569 17017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:21.429293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:51633)
0812 04:00:21.429470 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:21.429512 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
I20260812 04:00:22.209296 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:22.209031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:47005)
0812 04:00:22.209194 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:22.209244 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260812 04:00:22.292984 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:22.292708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:35333)
0812 04:00:22.292874 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:22.292917 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
I20260812 04:00:22.430364 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:22.430070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:39043)
0812 04:00:22.430248 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:22.430311 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":68,"threads_started":1}
I20260812 04:00:23.210049 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:23.209800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:37799)
0812 04:00:23.209949 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:23.209996 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260812 04:00:23.293682 17022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:23.293463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:55635)
0812 04:00:23.293596 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:23.293636 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260812 04:00:23.431275 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:23.431004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:60535)
0812 04:00:23.431182 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:23.431226 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260812 04:00:24.180765 17024 raft_consensus.cc:482] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:00:24.180902 17024 raft_consensus.cc:504] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:24.181131 17024 leader_election.cc:288] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713), b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077)
W20260812 04:00:24.181528 16552 leader_election.cc:341] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713): Not found: Tablet not found: 312b3cd97a3c462280d7c169eb4e5f70
W20260812 04:00:24.181631 16552 leader_election.cc:341] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077): Not found: Tablet not found: 312b3cd97a3c462280d7c169eb4e5f70
I20260812 04:00:24.181679 16552 leader_election.cc:302] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c8af0f13ad1411c9c3bc1553a4eb1cd; no voters: 637df5a1285446a29ffcc32ea70051a7, b528ed2b2162480a8bbcf21edfc18f6f
I20260812 04:00:24.181766 17024 raft_consensus.cc:2726] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 04:00:24.210788 17025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:24.210533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:49409)
0812 04:00:24.210701 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:24.210738 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260812 04:00:24.294530 17026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:24.294267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:56421)
0812 04:00:24.294424 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:24.294465 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":82,"threads_started":1}
I20260812 04:00:24.432024 17027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:24.431756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:59849)
0812 04:00:24.431901 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:24.431942 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260812 04:00:25.211733 17028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:25.211345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:47211)
0812 04:00:25.211574 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:25.211630 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":101,"threads_started":1}
I20260812 04:00:25.295356 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:25.295117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:42099)
0812 04:00:25.295254 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:25.295301 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260812 04:00:25.432834 17030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:25.432546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:51101)
0812 04:00:25.432735 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:25.432783 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":105,"threads_started":1}
I20260812 04:00:26.212484 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:26.212254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:47023)
0812 04:00:26.212389 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:26.212435 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260812 04:00:26.296175 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:26.295879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:50297)
0812 04:00:26.296077 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:26.296123 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":83,"threads_started":1}
I20260812 04:00:26.433715 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:26.433467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:38203)
0812 04:00:26.433619 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:26.433664 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260812 04:00:26.850057 17034 raft_consensus.cc:482] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:00:26.850198 17034 raft_consensus.cc:504] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "637df5a1285446a29ffcc32ea70051a7" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46713 } } peers { permanent_uuid: "b528ed2b2162480a8bbcf21edfc18f6f" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41077 } } peers { permanent_uuid: "1c8af0f13ad1411c9c3bc1553a4eb1cd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 36067 } }
I20260812 04:00:26.850441 17034 leader_election.cc:288] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713), b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077)
W20260812 04:00:26.850987 16552 leader_election.cc:341] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 637df5a1285446a29ffcc32ea70051a7 (127.31.43.195:46713): Not found: Tablet not found: 312b3cd97a3c462280d7c169eb4e5f70
W20260812 04:00:26.851075 16552 leader_election.cc:341] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b528ed2b2162480a8bbcf21edfc18f6f (127.31.43.194:41077): Not found: Tablet not found: 312b3cd97a3c462280d7c169eb4e5f70
I20260812 04:00:26.851099 16552 leader_election.cc:302] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c8af0f13ad1411c9c3bc1553a4eb1cd; no voters: 637df5a1285446a29ffcc32ea70051a7, b528ed2b2162480a8bbcf21edfc18f6f
I20260812 04:00:26.851187 17034 raft_consensus.cc:2726] T 312b3cd97a3c462280d7c169eb4e5f70 P 1c8af0f13ad1411c9c3bc1553a4eb1cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 04:00:27.213248 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:27.212933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:52551)
0812 04:00:27.213155 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:27.213197 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":71,"threads_started":1}
I20260812 04:00:27.296903 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:27.296678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:50857)
0812 04:00:27.296808 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:27.296852 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
I20260812 04:00:27.434494 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:27.434268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:39859)
0812 04:00:27.434412 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:27.434445 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260812 04:00:28.214051 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:28.213785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:39911)
0812 04:00:28.213950 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:28.213998 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260812 04:00:28.297683 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:28.297428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:57011)
0812 04:00:28.297584 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:28.297632 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260812 04:00:28.435312 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:28.435027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:49983)
0812 04:00:28.435201 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:28.435251 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260812 04:00:29.215023 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:29.214760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.193:53235)
0812 04:00:29.214912 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:29.214967 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260812 04:00:29.298623 17043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:29.298354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.194:60929)
0812 04:00:29.298523 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:29.298570 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
I20260812 04:00:29.436226 17044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:29.435968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40475 (local address 127.31.43.195:39853)
0812 04:00:29.436120 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:29.436174 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260812 04:00:29.437541 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 16495
I20260812 04:00:29.448614 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 16678
I20260812 04:00:29.456921 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 16831
2026-08-12T04:00:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18937 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:00:29 dist-test-slave-8tdl krb5kdc[17047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:00:29 dist-test-slave-8tdl krb5kdc[17047](info): set up 2 sockets
Aug 12 04:00:29 dist-test-slave-8tdl krb5kdc[17047](info): commencing operation
krb5kdc: starting...
W20260812 04:00:30.594774 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:00:30 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:00:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:00:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:00:30.729688 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:33795
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:33795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:30.815714 17062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:30.815958 17062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:30.815999 17062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:30.817193 17062 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 04:00:30.817260 17062 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 04:00:30.817281 17062 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 04:00:30.817308 17062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:00:30.817337 17062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:30.817368 17062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:30.817394 17062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:00:30.817413 17062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:00:30.819101 17062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:33795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:33795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:30.819386 17062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:30.819624 17062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:30.825160 17071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:30.825196 17062 server_base.cc:1034] running on GCE node
W20260812 04:00:30.825198 17068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:30.825141 17069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:30.825794 17062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:30.826061 17062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:30.827232 17062 hybrid_clock.cc:648] HybridClock initialized: now 1786507230827217 us; error 31 us; skew 500 ppm
Aug 12 04:00:30 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507230, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:30.834189 17062 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:00:30.834847 17062 webserver.cc:460] Webserver started at http://127.31.43.254:38639/ using document root <none> and password file <none>
I20260812 04:00:30.835150 17062 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:30.835218 17062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:30.835392 17062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:00:30.836724 17062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "512ac5d959fb4f628c94a91614366ead"
format_stamp: "Formatted at 2026-08-12 04:00:30 on dist-test-slave-8tdl"
server_key: "47b3ba48346e1899ee04a8aabd6c21a8"
server_key_iv: "5884426705c6f9d3b8957716800c331e"
server_key_version: "encryptionkey@0"
I20260812 04:00:30.837303 17062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "512ac5d959fb4f628c94a91614366ead"
format_stamp: "Formatted at 2026-08-12 04:00:30 on dist-test-slave-8tdl"
server_key: "47b3ba48346e1899ee04a8aabd6c21a8"
server_key_iv: "5884426705c6f9d3b8957716800c331e"
server_key_version: "encryptionkey@0"
I20260812 04:00:30.839028 17062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 04:00:30.839962 17078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:30.840167 17062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:30.840282 17062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "512ac5d959fb4f628c94a91614366ead"
format_stamp: "Formatted at 2026-08-12 04:00:30 on dist-test-slave-8tdl"
server_key: "47b3ba48346e1899ee04a8aabd6c21a8"
server_key_iv: "5884426705c6f9d3b8957716800c331e"
server_key_version: "encryptionkey@0"
I20260812 04:00:30.840363 17062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:30.876684 17062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:30.878927 17062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:30.879118 17062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:30.883525 17062 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:33795
I20260812 04:00:30.883579 17135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:33795 every 8 connection(s)
I20260812 04:00:30.883972 17062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:00:30.884876 17136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:30.886372 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 17062
I20260812 04:00:30.886498 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:00:30.886718 31919 external_mini_cluster.cc:1428] Setting key 6d9990621e4432b3c42e828097460b82
I20260812 04:00:30.887764 17136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Bootstrap starting.
I20260812 04:00:30.888394 17136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:30.888696 17136 log.cc:824] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:30.889582 17136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: No bootstrap required, opened a new log
I20260812 04:00:30.891937 17136 raft_consensus.cc:348] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } }
I20260812 04:00:30.892131 17136 raft_consensus.cc:374] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:30.892174 17136 raft_consensus.cc:729] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512ac5d959fb4f628c94a91614366ead, State: Initialized, Role: FOLLOWER
I20260812 04:00:30.892356 17136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } }
I20260812 04:00:30.892458 17136 raft_consensus.cc:388] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:00:30.892506 17136 raft_consensus.cc:482] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:00:30.892556 17136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:30.893196 17136 raft_consensus.cc:504] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } }
I20260812 04:00:30.893325 17136 leader_election.cc:302] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 512ac5d959fb4f628c94a91614366ead; no voters:
I20260812 04:00:30.893510 17136 leader_election.cc:288] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:00:30.893571 17142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:00:30.893756 17142 raft_consensus.cc:686] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 1 LEADER]: Becoming Leader. State: Replica: 512ac5d959fb4f628c94a91614366ead, State: Running, Role: LEADER
I20260812 04:00:30.893831 17136 sys_catalog.cc:564] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:00:30.893942 17142 consensus_queue.cc:237] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } }
I20260812 04:00:30.894326 17142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: SysCatalogTable state changed. Reason: New leader 512ac5d959fb4f628c94a91614366ead. Latest consensus state: current_term: 1 leader_uuid: "512ac5d959fb4f628c94a91614366ead" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } } }
Aug 12 04:00:30 dist-test-slave-8tdl krb5kdc[17047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:30.894618 17142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:30.894546 17144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "512ac5d959fb4f628c94a91614366ead" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } } }
I20260812 04:00:30.895123 17144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:30.895248 17154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:00:30.895790 17154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:00:30.897629 17154 catalog_manager.cc:1347] Generated new cluster ID: 3390aca7ed1b48b495a5ff5ac2d390e3
I20260812 04:00:30.897686 17154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:00:30.899173 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:30.887379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56680 (local address 127.31.43.254:33795)
0812 04:00:30.887588 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:30.887597 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:30.887623 (+ 26us) server_negotiation.cc:407] Connection header received
0812 04:00:30.888104 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:30.888112 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:30.888170 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:30.888319 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:30.889106 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:30.890494 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:30.891352 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:30.892038 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:30.892158 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:30.895129 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:30.895133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:30.895133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:30.895146 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:30.897215 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:30.897657 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:30.897660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:30.897661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:30.897765 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:30.897974 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:30.897975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:30.897975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:30.898252 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:30.898379 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:30.898828 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 04:00:30.898962 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260812 04:00:30.903060 17154 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:00:30.903656 17154 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:00:30.909361 17154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Generated new TSK 0
I20260812 04:00:30.909590 17154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 04:00:30.943332 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:33795
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:31.030339 17168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:31.030596 17168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:31.030638 17168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:31.031857 17168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:31.031934 17168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:31.031981 17168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:00:31.033531 17168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:31.033813 17168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:31.034052 17168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:31.039366 17173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:31.039417 17174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:31.039520 17168 server_base.cc:1034] running on GCE node
W20260812 04:00:31.040334 17179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:31.040585 17168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:31.040839 17168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:31.042053 17168 hybrid_clock.cc:648] HybridClock initialized: now 1786507231042034 us; error 29 us; skew 500 ppm
Aug 12 04:00:31 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:31.045154 17168 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 04:00:31.045639 17168 webserver.cc:460] Webserver started at http://127.31.43.193:45227/ using document root <none> and password file <none>
I20260812 04:00:31.045887 17168 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:31.045961 17168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:31.046115 17168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:00:31.047231 17168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "7c2e85f133ae41ceb4ec6549f9e8701e"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "a666a2cee3e1846234ac27aab8282215"
server_key_iv: "fa07fd0d109afc2c02942528189de6ff"
server_key_version: "encryptionkey@0"
I20260812 04:00:31.047596 17168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "7c2e85f133ae41ceb4ec6549f9e8701e"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "a666a2cee3e1846234ac27aab8282215"
server_key_iv: "fa07fd0d109afc2c02942528189de6ff"
server_key_version: "encryptionkey@0"
I20260812 04:00:31.049029 17168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:00:31.049813 17186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:31.050058 17168 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:31.050168 17168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "7c2e85f133ae41ceb4ec6549f9e8701e"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "a666a2cee3e1846234ac27aab8282215"
server_key_iv: "fa07fd0d109afc2c02942528189de6ff"
server_key_version: "encryptionkey@0"
I20260812 04:00:31.050240 17168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:31.063061 17168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:31.065083 17168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:31.065243 17168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:31.065475 17168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:31.065805 17168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:31.065858 17168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:31.065902 17168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:31.065938 17168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:31.072430 17168 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:45939
I20260812 04:00:31.072527 17317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:45939 every 8 connection(s)
I20260812 04:00:31.072825 17168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
Aug 12 04:00:31 dist-test-slave-8tdl krb5kdc[17047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:31.079993 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 17168
I20260812 04:00:31.080124 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 04:00:31.080349 31919 external_mini_cluster.cc:1428] Setting key 8c4c88e4c9cbae481e860d809202083f
I20260812 04:00:31.082890 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.073544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:50147 (local address 127.31.43.254:33795)
0812 04:00:31.073597 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:31.073599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:31.073816 (+ 217us) server_negotiation.cc:407] Connection header received
0812 04:00:31.074339 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:31.074341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:31.074354 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:31.074408 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:31.075482 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.076019 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.077014 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.077240 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.077323 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.079569 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:31.079570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:31.079571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:31.079579 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:31.081296 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:31.081783 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:31.081784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:31.081784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:31.081841 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:31.082168 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:31.082169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:31.082170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:31.082288 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:31.082353 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:31.082720 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 04:00:31.082820 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 04:00:31.083245 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.073503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:50147)
0812 04:00:31.073670 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:31.073714 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:31.074234 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:31.074498 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:31.074501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:31.075166 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:31.075368 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.075374 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.076123 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.076124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.076917 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.076919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.077338 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.077339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.077466 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.078053 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:31.078061 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:31.079470 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:31.081370 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:31.081379 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:31.081381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:31.081707 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:31.081907 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:31.081908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:31.081908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:31.082071 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:31.082360 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:31.082367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:31.082536 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 04:00:31.082652 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 04:00:31.082867 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260812 04:00:31.083763 17318 heartbeater.cc:347] Connected to a master server at 127.31.43.254:33795
I20260812 04:00:31.083855 17318 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:31.084079 17318 heartbeater.cc:511] Master 127.31.43.254:33795 requested a full tablet report, sending...
I20260812 04:00:31.084640 17096 ts_manager.cc:194] Registered new tserver with Master: 7c2e85f133ae41ceb4ec6549f9e8701e (127.31.43.193:45939)
I20260812 04:00:31.085552 17096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:50147
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 04:00:31.111171 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:33795
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:31.199874 17324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:31.200095 17324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:31.200119 17324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:31.201231 17324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:31.201270 17324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:31.201297 17324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:00:31.202828 17324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:31.203075 17324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:31.203279 17324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:31.208870 17330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:31.208911 17329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:31.208911 17332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:31.209169 17324 server_base.cc:1034] running on GCE node
I20260812 04:00:31.209373 17324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:31.209620 17324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:31.210747 17324 hybrid_clock.cc:648] HybridClock initialized: now 1786507231210732 us; error 41 us; skew 500 ppm
Aug 12 04:00:31 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:31.213359 17324 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 04:00:31.213831 17324 webserver.cc:460] Webserver started at http://127.31.43.194:34301/ using document root <none> and password file <none>
I20260812 04:00:31.214058 17324 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:31.214102 17324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:31.214195 17324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:00:31.215328 17324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "2a49b75f562b47e78bce2dbadc470d09"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "effb16b9e8dcac1116dedd824f7bdd3b"
server_key_iv: "a2e6fc880447bf5a9f96040ab6c1b6b1"
server_key_version: "encryptionkey@0"
I20260812 04:00:31.215658 17324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "2a49b75f562b47e78bce2dbadc470d09"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "effb16b9e8dcac1116dedd824f7bdd3b"
server_key_iv: "a2e6fc880447bf5a9f96040ab6c1b6b1"
server_key_version: "encryptionkey@0"
I20260812 04:00:31.217098 17324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:00:31.217921 17339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:31.218142 17324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:31.218241 17324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "2a49b75f562b47e78bce2dbadc470d09"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "effb16b9e8dcac1116dedd824f7bdd3b"
server_key_iv: "a2e6fc880447bf5a9f96040ab6c1b6b1"
server_key_version: "encryptionkey@0"
I20260812 04:00:31.218307 17324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:31.234339 17324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:31.237459 17324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:31.237596 17324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:31.237826 17324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:31.238190 17324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:31.238234 17324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:31.238282 17324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:31.238312 17324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:31.244879 17324 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:41829
I20260812 04:00:31.244975 17471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:41829 every 8 connection(s)
I20260812 04:00:31.245324 17324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:00:31.247766 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 17324
I20260812 04:00:31.247869 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 04:00:31.248106 31919 external_mini_cluster.cc:1428] Setting key c5d13c93c2f6863b3cf4f7a86551f711
Aug 12 04:00:31 dist-test-slave-8tdl krb5kdc[17047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:31.255810 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.246184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:60641 (local address 127.31.43.254:33795)
0812 04:00:31.246249 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:31.246251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:31.246494 (+ 243us) server_negotiation.cc:407] Connection header received
0812 04:00:31.247084 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:31.247085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:31.247099 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:31.247159 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:31.248296 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.248931 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.249933 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.250135 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.250222 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.252673 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:31.252675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:31.252675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:31.252683 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:31.254323 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:31.254792 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:31.254793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:31.254794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:31.254855 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:31.255124 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:31.255125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:31.255125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:31.255245 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:31.255313 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:31.255622 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 04:00:31.255714 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 04:00:31.256048 17474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.246178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:60641)
0812 04:00:31.246356 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:31.246391 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:31.246961 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:31.247229 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:31.247233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:31.247909 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:31.248185 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.248192 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.249027 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.249029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.249838 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.249840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.250241 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.250242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.250375 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.250971 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:31.250979 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:31.252567 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:31.254428 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:31.254437 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:31.254439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:31.254728 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:31.254933 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:31.254934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:31.254935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:31.255051 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:31.255316 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:31.255324 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:31.255481 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 04:00:31.255586 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 04:00:31.255757 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260812 04:00:31.256531 17472 heartbeater.cc:347] Connected to a master server at 127.31.43.254:33795
I20260812 04:00:31.256628 17472 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:31.256834 17472 heartbeater.cc:511] Master 127.31.43.254:33795 requested a full tablet report, sending...
I20260812 04:00:31.257268 17096 ts_manager.cc:194] Registered new tserver with Master: 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829)
I20260812 04:00:31.257869 17096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:60641
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 04:00:31.281781 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:33795
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:31.368506 17477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:31.368732 17477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:31.368758 17477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:31.369949 17477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:31.369995 17477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:31.370021 17477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:00:31.371599 17477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:31.371865 17477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:31.372150 17477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:31.377537 17486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:31.377578 17477 server_base.cc:1034] running on GCE node
W20260812 04:00:31.377558 17482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:31.377570 17484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:31.378022 17477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:31.378328 17477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:31.379484 17477 hybrid_clock.cc:648] HybridClock initialized: now 1786507231379471 us; error 45 us; skew 500 ppm
Aug 12 04:00:31 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:31.382282 17477 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 04:00:31.382815 17477 webserver.cc:460] Webserver started at http://127.31.43.195:41503/ using document root <none> and password file <none>
I20260812 04:00:31.383060 17477 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:31.383108 17477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:31.383270 17477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:00:31.384366 17477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "9ce616a372a54fe98ed432a263a734b4"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "93b698ce9a7ef15433a31270be5e097a"
server_key_iv: "b44ea206e028e0960cc1b6f17acbeac9"
server_key_version: "encryptionkey@0"
I20260812 04:00:31.384747 17477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "9ce616a372a54fe98ed432a263a734b4"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "93b698ce9a7ef15433a31270be5e097a"
server_key_iv: "b44ea206e028e0960cc1b6f17acbeac9"
server_key_version: "encryptionkey@0"
I20260812 04:00:31.386214 17477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:00:31.387180 17495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:31.387367 17477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 04:00:31.387475 17477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "9ce616a372a54fe98ed432a263a734b4"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "93b698ce9a7ef15433a31270be5e097a"
server_key_iv: "b44ea206e028e0960cc1b6f17acbeac9"
server_key_version: "encryptionkey@0"
I20260812 04:00:31.387616 17477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:31.412056 17477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:31.414357 17477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:31.414565 17477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:31.414875 17477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:31.415230 17477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:31.415287 17477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:31.415329 17477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:31.415356 17477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:31.422079 17477 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:44585
I20260812 04:00:31.422124 17618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:44585 every 8 connection(s)
I20260812 04:00:31.422466 17477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 04:00:31.428157 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 17477
I20260812 04:00:31.428300 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 04:00:31.428499 31919 external_mini_cluster.cc:1428] Setting key b99cb2e4b054db7e1989385a94742350
Aug 12 04:00:31 dist-test-slave-8tdl krb5kdc[17047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:31.432914 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.423368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:58739 (local address 127.31.43.254:33795)
0812 04:00:31.423655 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:31.423657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:31.423670 (+ 13us) server_negotiation.cc:407] Connection header received
0812 04:00:31.424141 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:31.424142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:31.424153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:31.424226 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:31.425222 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.425748 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.426958 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.427194 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.427270 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.429842 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:31.429843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:31.429844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:31.429852 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:31.431312 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:31.431794 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:31.431795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:31.431795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:31.431875 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:31.432154 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:31.432155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:31.432156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:31.432298 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:31.432383 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:31.432710 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 04:00:31.432808 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 04:00:31.433123 17621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.423331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:58739)
0812 04:00:31.423499 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:31.423545 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:31.424048 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:31.424285 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:31.424288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:31.424910 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:31.425127 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.425133 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.425839 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.425840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.426853 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.426857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.427292 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.427293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.427401 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.428018 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:31.428028 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:31.429746 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:31.431423 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:31.431432 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:31.431435 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:31.431707 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:31.431951 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:31.431952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:31.431952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:31.432079 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:31.432373 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:31.432380 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:31.432551 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 04:00:31.432664 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 04:00:31.432852 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260812 04:00:31.433679 17619 heartbeater.cc:347] Connected to a master server at 127.31.43.254:33795
I20260812 04:00:31.433805 17619 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:31.434044 17619 heartbeater.cc:511] Master 127.31.43.254:33795 requested a full tablet report, sending...
I20260812 04:00:31.434435 17096 ts_manager.cc:194] Registered new tserver with Master: 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585)
I20260812 04:00:31.434928 17096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:58739
I20260812 04:00:31.440400 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 04:00:31.448590 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.441843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56694 (local address 127.31.43.254:33795)
0812 04:00:31.441902 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:31.441904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:31.442087 (+ 183us) server_negotiation.cc:407] Connection header received
0812 04:00:31.442147 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:31.442148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:31.442158 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:31.442218 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:31.442900 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.443500 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.444472 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.444665 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.444733 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.446184 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:31.446185 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:31.446185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:31.446192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:31.447588 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:31.447916 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:31.447916 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:31.447917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:31.447971 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:31.448143 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:31.448144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:31.448144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:31.448292 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:31.448378 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:31.448455 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 04:00:31.448521 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 04:00:31.450320 17096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 04:00:31.452159 17096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 04:00:31.459409 17635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.454853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.31.43.194:41829)
0812 04:00:31.455121 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:31.455124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:31.455681 (+ 557us) server_negotiation.cc:407] Connection header received
0812 04:00:31.455728 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:31.455731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:31.455745 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:31.455828 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:31.455999 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.457029 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.458182 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.458967 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.459050 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.459114 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:31.459295 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 04:00:31.459331 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":121,"threads_started":1}
I20260812 04:00:31.459466 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.455238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:41829 (local address 127.0.0.1:52186)
0812 04:00:31.455560 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:31.455569 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:31.455616 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:31.455838 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:31.455839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:31.455842 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:31.455904 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.455906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.457123 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.457124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.458085 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.458088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.459074 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.459075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.459170 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.459187 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 04:00:31.459281 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 04:00:31.459341 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260812 04:00:31.460047 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.454901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.31.43.195:44585)
0812 04:00:31.455103 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:31.455107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:31.455460 (+ 353us) server_negotiation.cc:407] Connection header received
0812 04:00:31.455607 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:31.455611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:31.455629 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:31.455686 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:31.455951 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.456657 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.458260 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.458978 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.459056 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.459114 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:31.459876 (+ 762us) server_negotiation.cc:299] Negotiation successful
0812 04:00:31.459929 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":86,"threads_started":1}
I20260812 04:00:31.460198 17631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.454705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45939 (local address 127.0.0.1:58510)
0812 04:00:31.454878 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:31.454918 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:31.455002 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:31.455424 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:31.455427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:31.455430 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:31.455502 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.455509 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.456876 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.456877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.458182 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.458185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.459906 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.459907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.459967 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.459971 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:00:31.460033 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 04:00:31.460088 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":48,"thread_start_us":60,"threads_started":1}
I20260812 04:00:31.460249 17639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.455084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58510 (local address 127.31.43.193:45939)
0812 04:00:31.455251 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:31.455254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:31.455262 (+ 8us) server_negotiation.cc:407] Connection header received
0812 04:00:31.455297 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:31.455298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:31.455311 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:31.455400 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:31.455610 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.456771 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.458816 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.459715 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.459789 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.459853 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:31.460114 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 04:00:31.460168 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260812 04:00:31.460358 17397 tablet_service.cc:1467] Processing CreateTablet for tablet c2fd51bab32e44c98342fbbfd3c94b6a (DEFAULT_TABLE table=test-table [id=6839be1215144e34af9e6783cba5aac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:31.460668 17640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.454705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:44585 (local address 127.0.0.1:37636)
0812 04:00:31.455382 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:31.455394 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:31.455503 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:31.455825 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:31.455826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:31.455828 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:31.455890 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.455892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.456761 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.456762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.458182 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.458185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.459679 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.459680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.459756 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.459761 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 04:00:31.460514 (+ 753us) client_negotiation.cc:240] Negotiation successful
0812 04:00:31.460587 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":51,"thread_start_us":97,"threads_started":1}
I20260812 04:00:31.460777 17397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2fd51bab32e44c98342fbbfd3c94b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:31.461172 17241 tablet_service.cc:1467] Processing CreateTablet for tablet c2fd51bab32e44c98342fbbfd3c94b6a (DEFAULT_TABLE table=test-table [id=6839be1215144e34af9e6783cba5aac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:31.461508 17241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2fd51bab32e44c98342fbbfd3c94b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:31.461428 17547 tablet_service.cc:1467] Processing CreateTablet for tablet c2fd51bab32e44c98342fbbfd3c94b6a (DEFAULT_TABLE table=test-table [id=6839be1215144e34af9e6783cba5aac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:31.461748 17547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2fd51bab32e44c98342fbbfd3c94b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:31.464296 17641 tablet_bootstrap.cc:492] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Bootstrap starting.
I20260812 04:00:31.465111 17643 tablet_bootstrap.cc:492] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4: Bootstrap starting.
I20260812 04:00:31.465113 17642 tablet_bootstrap.cc:492] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09: Bootstrap starting.
I20260812 04:00:31.465166 17641 tablet_bootstrap.cc:654] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:31.465667 17642 tablet_bootstrap.cc:654] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:31.465895 17641 log.cc:824] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:31.465924 17643 tablet_bootstrap.cc:654] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:31.466224 17642 log.cc:824] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:31.466596 17643 log.cc:824] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:31.466893 17641 tablet_bootstrap.cc:492] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: No bootstrap required, opened a new log
I20260812 04:00:31.466957 17642 tablet_bootstrap.cc:492] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09: No bootstrap required, opened a new log
I20260812 04:00:31.466980 17641 ts_tablet_manager.cc:1397] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 04:00:31.467043 17642 ts_tablet_manager.cc:1397] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 04:00:31.467466 17643 tablet_bootstrap.cc:492] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4: No bootstrap required, opened a new log
I20260812 04:00:31.467687 17643 ts_tablet_manager.cc:1397] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 04:00:31.468783 17641 raft_consensus.cc:348] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } } peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } }
I20260812 04:00:31.468964 17641 raft_consensus.cc:374] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:31.469035 17641 raft_consensus.cc:729] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2e85f133ae41ceb4ec6549f9e8701e, State: Initialized, Role: FOLLOWER
I20260812 04:00:31.469180 17641 consensus_queue.cc:260] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } } peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } }
I20260812 04:00:31.469242 17642 raft_consensus.cc:348] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } } peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } }
I20260812 04:00:31.469453 17642 raft_consensus.cc:374] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:31.469498 17642 raft_consensus.cc:729] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a49b75f562b47e78bce2dbadc470d09, State: Initialized, Role: FOLLOWER
I20260812 04:00:31.469538 17318 heartbeater.cc:503] Master 127.31.43.254:33795 was elected leader, sending a full tablet report...
I20260812 04:00:31.469606 17642 consensus_queue.cc:260] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } } peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } }
I20260812 04:00:31.469527 17641 ts_tablet_manager.cc:1428] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 04:00:31.469679 17643 raft_consensus.cc:348] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } } peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } }
I20260812 04:00:31.469841 17643 raft_consensus.cc:374] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:31.469877 17642 ts_tablet_manager.cc:1428] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 04:00:31.469877 17643 raft_consensus.cc:729] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ce616a372a54fe98ed432a263a734b4, State: Initialized, Role: FOLLOWER
I20260812 04:00:31.470093 17643 consensus_queue.cc:260] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } } peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } }
I20260812 04:00:31.470423 17472 heartbeater.cc:503] Master 127.31.43.254:33795 was elected leader, sending a full tablet report...
I20260812 04:00:31.470425 17619 heartbeater.cc:503] Master 127.31.43.254:33795 was elected leader, sending a full tablet report...
I20260812 04:00:31.470405 17643 ts_tablet_manager.cc:1428] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260812 04:00:31.495971 17473 tablet.cc:2369] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:00:31.516350 17649 raft_consensus.cc:482] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:00:31.516577 17649 raft_consensus.cc:504] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } } peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } }
I20260812 04:00:31.517078 17649 leader_election.cc:288] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c2e85f133ae41ceb4ec6549f9e8701e (127.31.43.193:45939), 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829)
I20260812 04:00:31.520293 17635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.517457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:42395 (local address 127.31.43.194:41829)
0812 04:00:31.517522 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:31.517524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:31.517569 (+ 45us) server_negotiation.cc:407] Connection header received
0812 04:00:31.517603 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:31.517604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:31.517616 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:31.517686 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:31.517898 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.518585 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.519560 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.519955 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.520037 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.520047 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:31.520185 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 04:00:31.520221 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 04:00:31.520293 17621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.517336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:41829 (local address 127.31.43.195:42395)
0812 04:00:31.517435 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:31.517449 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:31.517501 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:31.517729 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:31.517730 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:31.517733 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:31.517816 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.517819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.518664 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.518665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.519466 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.519468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.520035 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.520036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.520113 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.520116 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:00:31.520180 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 04:00:31.520221 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":20}
I20260812 04:00:31.520390 17639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.517469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:56645 (local address 127.31.43.193:45939)
0812 04:00:31.517522 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:31.517524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:31.517712 (+ 188us) server_negotiation.cc:407] Connection header received
0812 04:00:31.517746 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:31.517747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:31.517757 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:31.517822 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:31.518006 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.518599 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.519560 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.520024 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.520126 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.520136 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:31.520287 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 04:00:31.520324 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 04:00:31.520413 17650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.517337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45939 (local address 127.31.43.195:56645)
0812 04:00:31.517599 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:31.517611 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:31.517685 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:31.517841 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:31.517842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:31.517844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:31.517924 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.517926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.518705 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.518706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.519469 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:31.519471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.520131 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.520132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:31.520203 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.520206 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:00:31.520279 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 04:00:31.520318 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":25,"thread_start_us":136,"threads_started":1}
I20260812 04:00:31.520761 17419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2fd51bab32e44c98342fbbfd3c94b6a" candidate_uuid: "9ce616a372a54fe98ed432a263a734b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a49b75f562b47e78bce2dbadc470d09" is_pre_election: true
I20260812 04:00:31.520939 17419 raft_consensus.cc:2454] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ce616a372a54fe98ed432a263a734b4 in term 0.
I20260812 04:00:31.521194 17264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2fd51bab32e44c98342fbbfd3c94b6a" candidate_uuid: "9ce616a372a54fe98ed432a263a734b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" is_pre_election: true
I20260812 04:00:31.521242 17496 leader_election.cc:302] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a49b75f562b47e78bce2dbadc470d09, 9ce616a372a54fe98ed432a263a734b4; no voters:
I20260812 04:00:31.521354 17264 raft_consensus.cc:2454] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ce616a372a54fe98ed432a263a734b4 in term 0.
I20260812 04:00:31.521417 17649 raft_consensus.cc:2781] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 04:00:31.521495 17649 raft_consensus.cc:482] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 04:00:31.521533 17649 raft_consensus.cc:3037] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:31.522488 17649 raft_consensus.cc:504] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } } peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } }
I20260812 04:00:31.522645 17649 leader_election.cc:288] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7c2e85f133ae41ceb4ec6549f9e8701e (127.31.43.193:45939), 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829)
I20260812 04:00:31.522835 17419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2fd51bab32e44c98342fbbfd3c94b6a" candidate_uuid: "9ce616a372a54fe98ed432a263a734b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a49b75f562b47e78bce2dbadc470d09"
I20260812 04:00:31.522874 17264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2fd51bab32e44c98342fbbfd3c94b6a" candidate_uuid: "9ce616a372a54fe98ed432a263a734b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e"
I20260812 04:00:31.522946 17419 raft_consensus.cc:3037] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:31.522962 17264 raft_consensus.cc:3037] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:31.523705 17264 raft_consensus.cc:2454] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ce616a372a54fe98ed432a263a734b4 in term 1.
I20260812 04:00:31.523789 17419 raft_consensus.cc:2454] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ce616a372a54fe98ed432a263a734b4 in term 1.
I20260812 04:00:31.524019 17498 leader_election.cc:302] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c2e85f133ae41ceb4ec6549f9e8701e, 9ce616a372a54fe98ed432a263a734b4; no voters:
I20260812 04:00:31.524138 17649 raft_consensus.cc:2781] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:00:31.524334 17649 raft_consensus.cc:686] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 1 LEADER]: Becoming Leader. State: Replica: 9ce616a372a54fe98ed432a263a734b4, State: Running, Role: LEADER
I20260812 04:00:31.524451 17649 consensus_queue.cc:237] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } } peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } }
I20260812 04:00:31.525497 17094 catalog_manager.cc:5495] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195). New cstate: current_term: 1 leader_uuid: "9ce616a372a54fe98ed432a263a734b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260812 04:00:31.537899 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.535388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37648 (local address 127.31.43.195:44585)
0812 04:00:31.535460 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:31.535462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:31.535471 (+ 9us) server_negotiation.cc:407] Connection header received
0812 04:00:31.535544 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:31.535545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:31.535562 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:31.535633 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:00:31.535774 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.536287 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.537084 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:31.537281 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:31.537363 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:31.537539 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:00:31.537639 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:00:31.537744 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:31.537764 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 04:00:31.537787 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 04:00:31.539963 17419 raft_consensus.cc:1261] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09 [term 1 FOLLOWER]: Refusing update from remote peer 9ce616a372a54fe98ed432a263a734b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:00:31.539970 17264 raft_consensus.cc:1261] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Refusing update from remote peer 9ce616a372a54fe98ed432a263a734b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:00:31.540397 17649 consensus_queue.cc:1035] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:00:31.540593 17651 consensus_queue.cc:1035] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:00:31.543561 17656 mvcc.cc:205] Tried to move back new op lower bound from 7317533620384772096 to 7317533620325089280. Current Snapshot: MvccSnapshot[applied={T|T < 7317533620384772096}]
I20260812 04:00:31.546114 17094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:00:31.546226 17094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:00:31.547003 17094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Sending DeleteTablet for 3 replicas of tablet c2fd51bab32e44c98342fbbfd3c94b6a
I20260812 04:00:31.547369 17547 tablet_service.cc:1514] Processing DeleteTablet for tablet c2fd51bab32e44c98342fbbfd3c94b6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:31 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:37636
I20260812 04:00:31.547386 17397 tablet_service.cc:1514] Processing DeleteTablet for tablet c2fd51bab32e44c98342fbbfd3c94b6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:31 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:52186
I20260812 04:00:31.547411 17241 tablet_service.cc:1514] Processing DeleteTablet for tablet c2fd51bab32e44c98342fbbfd3c94b6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:31 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:58510
I20260812 04:00:31.547649 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 17168
I20260812 04:00:31.547654 17665 tablet_replica.cc:330] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09: stopping tablet replica
I20260812 04:00:31.547753 17665 raft_consensus.cc:2229] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:00:31.547843 17665 raft_consensus.cc:2258] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:00:31.547988 17666 tablet_replica.cc:330] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4: stopping tablet replica
I20260812 04:00:31.548059 17666 raft_consensus.cc:2229] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 04:00:31.548127 17666 raft_consensus.cc:2258] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:00:31.550544 17666 ts_tablet_manager.cc:1905] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:00:31.552677 17665 ts_tablet_manager.cc:1905] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:00:31.552943 17666 ts_tablet_manager.cc:1918] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:00:31.553082 17666 log.cc:1196] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/c2fd51bab32e44c98342fbbfd3c94b6a
I20260812 04:00:31.553503 17666 ts_tablet_manager.cc:1939] T c2fd51bab32e44c98342fbbfd3c94b6a P 9ce616a372a54fe98ed432a263a734b4: Deleting consensus metadata
I20260812 04:00:31.554060 17082 catalog_manager.cc:4841] TS 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585): tablet c2fd51bab32e44c98342fbbfd3c94b6a (table test-table [id=6839be1215144e34af9e6783cba5aac9]) successfully deleted
I20260812 04:00:31.554600 17665 ts_tablet_manager.cc:1918] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:00:31.554708 17665 log.cc:1196] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/c2fd51bab32e44c98342fbbfd3c94b6a
I20260812 04:00:31.555094 17665 ts_tablet_manager.cc:1939] T c2fd51bab32e44c98342fbbfd3c94b6a P 2a49b75f562b47e78bce2dbadc470d09: Deleting consensus metadata
I20260812 04:00:31.555559 17079 catalog_manager.cc:4841] TS 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829): tablet c2fd51bab32e44c98342fbbfd3c94b6a (table test-table [id=6839be1215144e34af9e6783cba5aac9]) successfully deleted
W20260812 04:00:31.558760 17081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:45939 (error 108)
I20260812 04:00:31.558974 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 17324
I20260812 04:00:31.567888 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:31.567672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45939 (local address 127.0.0.1:58526)
0812 04:00:31.567756 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:31.567829 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:45939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 04:00:31.568249 17081 catalog_manager.cc:4584] TS 7c2e85f133ae41ceb4ec6549f9e8701e (127.31.43.193:45939): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c2fd51bab32e44c98342fbbfd3c94b6a: Network error: Client connection negotiation failed: client connection to 127.31.43.193:45939: connect: Connection refused (error 111)
I20260812 04:00:31.576763 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 17477
I20260812 04:00:31.584434 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 17062
I20260812 04:00:51.594030 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:33795
--webserver_interface=127.31.43.254
--webserver_port=38639
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:33795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:00:51.681248 17670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:51.681504 17670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:51.681543 17670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:51.682754 17670 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 04:00:51.682829 17670 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 04:00:51.682850 17670 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 04:00:51.682870 17670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:00:51.682910 17670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:51.682937 17670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:51.682966 17670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:00:51.682994 17670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:00:51.684576 17670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:33795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:33795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=38639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:51.684878 17670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:51.685125 17670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:51.695493 17676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:51.695529 17670 server_base.cc:1034] running on GCE node
W20260812 04:00:51.695497 17677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:51.695530 17679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:51.696242 17670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:51.696602 17670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:51.697818 17670 hybrid_clock.cc:648] HybridClock initialized: now 1786507251697811 us; error 53 us; skew 500 ppm
Aug 12 04:00:51 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507251, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:51.708930 17670 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:00:51.709429 17670 webserver.cc:460] Webserver started at http://127.31.43.254:38639/ using document root <none> and password file <none>
I20260812 04:00:51.709735 17670 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:51.709815 17670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:51.711508 17670 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:00:51.712249 17686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:51.712522 17670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:51.712631 17670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "512ac5d959fb4f628c94a91614366ead"
format_stamp: "Formatted at 2026-08-12 04:00:30 on dist-test-slave-8tdl"
server_key: "47b3ba48346e1899ee04a8aabd6c21a8"
server_key_iv: "5884426705c6f9d3b8957716800c331e"
server_key_version: "encryptionkey@0"
I20260812 04:00:51.713146 17670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:51.736191 17670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:51.738413 17670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:51.738584 17670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:51.743412 17670 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:33795
I20260812 04:00:51.743455 17742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:33795 every 8 connection(s)
I20260812 04:00:51.743916 17670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:00:51.745743 17743 sys_catalog.cc:263] Verifying existing consensus state
I20260812 04:00:51.746557 17743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Bootstrap starting.
I20260812 04:00:51.750982 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 17670
I20260812 04:00:51.751348 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:45939
--local_ip_for_outbound_sockets=127.31.43.193
--tserver_master_addrs=127.31.43.254:33795
--webserver_port=45227
--webserver_interface=127.31.43.193
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:00:51.757366 17743 log.cc:824] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:51.759831 17743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 04:00:51.760241 17743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Bootstrap complete.
I20260812 04:00:51.763191 17743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } }
I20260812 04:00:51.763513 17743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512ac5d959fb4f628c94a91614366ead, State: Initialized, Role: FOLLOWER
I20260812 04:00:51.763769 17743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } }
I20260812 04:00:51.763871 17743 raft_consensus.cc:388] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:00:51.763944 17743 raft_consensus.cc:482] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:00:51.764010 17743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 1 FOLLOWER]: Advancing to term 2
I20260812 04:00:51.765285 17743 raft_consensus.cc:504] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } }
I20260812 04:00:51.765436 17743 leader_election.cc:302] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 512ac5d959fb4f628c94a91614366ead; no voters:
I20260812 04:00:51.765704 17743 leader_election.cc:288] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 04:00:51.765842 17747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 2 FOLLOWER]: Leader election won for term 2
I20260812 04:00:51.766038 17743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:00:51.766080 17747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 2 LEADER]: Becoming Leader. State: Replica: 512ac5d959fb4f628c94a91614366ead, State: Running, Role: LEADER
I20260812 04:00:51.766379 17747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } }
I20260812 04:00:51.766774 17749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: SysCatalogTable state changed. Reason: New leader 512ac5d959fb4f628c94a91614366ead. Latest consensus state: current_term: 2 leader_uuid: "512ac5d959fb4f628c94a91614366ead" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } } }
I20260812 04:00:51.766850 17749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:51.767396 17748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "512ac5d959fb4f628c94a91614366ead" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } } }
I20260812 04:00:51.767532 17748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: This master's current role is: LEADER
I20260812 04:00:51.767745 17756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:00:51.768883 17756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:00:51.769032 17756 catalog_manager.cc:1259] Loaded cluster ID: 3390aca7ed1b48b495a5ff5ac2d390e3
I20260812 04:00:51.769078 17756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:00:51.770303 17756 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:00:51.770535 17756 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Loaded TSK: 0
I20260812 04:00:51.790177 17756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Generated new TSK 1
I20260812 04:00:51.790387 17756 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 04:00:51.847512 17745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:51.847761 17745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:51.847803 17745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:51.849046 17745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:51.849123 17745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:51.849170 17745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:00:51.850795 17745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:45939
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=45227
--tserver_master_addrs=127.31.43.254:33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:51.851078 17745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:51.851329 17745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:51.858094 17776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:51.858115 17774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:51.858312 17745 server_base.cc:1034] running on GCE node
W20260812 04:00:51.858101 17773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:51.858613 17745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:51.858922 17745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:51.860111 17745 hybrid_clock.cc:648] HybridClock initialized: now 1786507251860090 us; error 42 us; skew 500 ppm
Aug 12 04:00:51 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507251, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:51.863240 17745 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 04:00:51.863876 17745 webserver.cc:460] Webserver started at http://127.31.43.193:45227/ using document root <none> and password file <none>
I20260812 04:00:51.864172 17745 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:51.864274 17745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:51.866354 17745 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 04:00:51.867334 17785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:51.867524 17745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 04:00:51.867750 17745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "7c2e85f133ae41ceb4ec6549f9e8701e"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "a666a2cee3e1846234ac27aab8282215"
server_key_iv: "fa07fd0d109afc2c02942528189de6ff"
server_key_version: "encryptionkey@0"
I20260812 04:00:51.868270 17745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:51.905638 17745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:51.909018 17745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:51.909233 17745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:51.909577 17745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:51.910207 17794 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 04:00:51.911971 17745 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 04:00:51.912035 17745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 04:00:51.912072 17745 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 04:00:51.912845 17745 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 04:00:51.912907 17745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 04:00:51.912948 17794 tablet_bootstrap.cc:492] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Bootstrap starting.
I20260812 04:00:51.921769 17745 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:45939
I20260812 04:00:51.921972 17913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:45939 every 8 connection(s)
I20260812 04:00:51.922463 17745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 04:00:51.926144 17794 log.cc:824] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:51.928673 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 17745
I20260812 04:00:51.929019 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:41829
--local_ip_for_outbound_sockets=127.31.43.194
--tserver_master_addrs=127.31.43.254:33795
--webserver_port=34301
--webserver_interface=127.31.43.194
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:00:51.929179 17794 tablet_bootstrap.cc:492] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 04:00:51.929533 17794 tablet_bootstrap.cc:492] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Bootstrap complete.
I20260812 04:00:51.931071 17794 ts_tablet_manager.cc:1397] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.007s
I20260812 04:00:51.932637 17794 raft_consensus.cc:348] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } } peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } }
I20260812 04:00:51.933049 17794 raft_consensus.cc:729] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2e85f133ae41ceb4ec6549f9e8701e, State: Initialized, Role: FOLLOWER
I20260812 04:00:51.933245 17794 consensus_queue.cc:260] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } } peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } }
I20260812 04:00:51.933646 17794 ts_tablet_manager.cc:1428] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
Aug 12 04:00:51 dist-test-slave-8tdl krb5kdc[17047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507251, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:51.939805 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:51.923535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:38033 (local address 127.31.43.254:33795)
0812 04:00:51.926554 (+ 3019us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:51.926565 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:51.926624 (+ 59us) server_negotiation.cc:407] Connection header received
0812 04:00:51.926843 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:51.926850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:51.926914 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:51.927095 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:51.927798 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:51.929069 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:51.930910 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:51.931656 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:51.931783 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:51.935096 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:51.935100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:51.935101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:51.935116 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:51.937444 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:51.938124 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:51.938129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:51.938130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:51.938268 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:51.938628 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:51.938629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:51.938629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:51.938953 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:51.939045 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:51.939523 (+ 478us) server_negotiation.cc:299] Negotiation successful
0812 04:00:51.939645 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2958,"thread_start_us":93,"threads_started":1}
I20260812 04:00:51.940110 17916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:51.923371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:38033)
0812 04:00:51.923631 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:51.923688 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:51.924406 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:51.927129 (+ 2723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:51.927134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:51.927445 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:51.927657 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:51.927663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:51.929248 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:51.929250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:51.930779 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:51.930784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:51.931931 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:51.931934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:51.932114 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:51.932798 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:51.932811 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:51.934962 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:51.937562 (+ 2600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:51.937573 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:51.937575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:51.938000 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:51.938364 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:51.938365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:51.938365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:51.938542 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:51.939087 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:51.939096 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:51.939298 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 04:00:51.939452 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 04:00:51.939686 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":132,"threads_started":1}
I20260812 04:00:51.940984 17914 heartbeater.cc:347] Connected to a master server at 127.31.43.254:33795
I20260812 04:00:51.941108 17914 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:51.941344 17914 heartbeater.cc:511] Master 127.31.43.254:33795 requested a full tablet report, sending...
I20260812 04:00:51.942328 17703 ts_manager.cc:194] Registered new tserver with Master: 7c2e85f133ae41ceb4ec6549f9e8701e (127.31.43.193:45939)
I20260812 04:00:51.943583 17703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:38033
I20260812 04:00:51.944806 17914 heartbeater.cc:503] Master 127.31.43.254:33795 was elected leader, sending a full tablet report...
Aug 12 04:00:51 dist-test-slave-8tdl krb5kdc[17047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507251, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.193@KRBTEST.COM
I20260812 04:00:51.952703 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:51.943144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51668 (local address 127.31.43.193:45939)
0812 04:00:51.943782 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:51.943786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:51.943798 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:00:51.943840 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:51.943842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:51.943858 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:51.943942 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:51.944441 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:51.945306 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:51.946619 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:51.946961 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:51.947026 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:51.949409 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:51.949417 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:51.949417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:51.949424 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:51.951147 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:51.951652 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:51.951655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:51.951656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:51.951747 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:51.952065 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:51.952065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:51.952066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:51.952229 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:51.952302 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:51.952550 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 04:00:51.952632 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":104,"threads_started":1}
I20260812 04:00:51.952761 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:51.943051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45939 (local address 127.0.0.1:51668)
0812 04:00:51.943292 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:51.943323 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:51.943388 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:51.943972 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:51.943978 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:51.944293 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:51.944344 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:51.944347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:51.945460 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:51.945461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:51.946529 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:51.946532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:51.947039 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:51.947040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:51.947127 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:51.947648 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:51.947654 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:51.949313 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:51.951229 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:51.951237 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:51.951240 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:51.951536 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:51.951841 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:51.951842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:51.951843 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:51.951938 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:51.952310 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:51.952318 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:51.952400 (+ 82us) client_negotiation.cc:769] Sending connection context
0812 04:00:51.952504 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 04:00:51.952643 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":103,"threads_started":1}
I20260812 04:00:51.953313 17821 tablet_service.cc:1514] Processing DeleteTablet for tablet c2fd51bab32e44c98342fbbfd3c94b6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:31 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:51668
I20260812 04:00:51.954035 17925 tablet_replica.cc:330] stopping tablet replica
I20260812 04:00:51.954174 17925 raft_consensus.cc:2229] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:00:51.954263 17925 raft_consensus.cc:2258] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:00:51.954623 17925 ts_tablet_manager.cc:1905] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:00:51.957065 17925 ts_tablet_manager.cc:1918] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:00:51.957171 17925 log.cc:1196] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal/wals/c2fd51bab32e44c98342fbbfd3c94b6a
I20260812 04:00:51.957420 17925 ts_tablet_manager.cc:1939] T c2fd51bab32e44c98342fbbfd3c94b6a P 7c2e85f133ae41ceb4ec6549f9e8701e: Deleting consensus metadata
I20260812 04:00:51.957981 17689 catalog_manager.cc:4841] TS 7c2e85f133ae41ceb4ec6549f9e8701e (127.31.43.193:45939): tablet c2fd51bab32e44c98342fbbfd3c94b6a (table test-table [id=6839be1215144e34af9e6783cba5aac9]) successfully deleted
W20260812 04:00:52.028489 17919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:52.028712 17919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:52.028735 17919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:52.029939 17919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:52.029987 17919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:52.030015 17919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:00:52.031594 17919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:41829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=34301
--tserver_master_addrs=127.31.43.254:33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:52.031867 17919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:52.032116 17919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:52.037622 17932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:52.037670 17933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:52.037626 17935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:52.037909 17919 server_base.cc:1034] running on GCE node
I20260812 04:00:52.038136 17919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:52.038360 17919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:52.039516 17919 hybrid_clock.cc:648] HybridClock initialized: now 1786507252039503 us; error 31 us; skew 500 ppm
Aug 12 04:00:52 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507252, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:52.042577 17919 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 04:00:52.043285 17919 webserver.cc:460] Webserver started at http://127.31.43.194:34301/ using document root <none> and password file <none>
I20260812 04:00:52.043561 17919 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:52.043666 17919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:52.045249 17919 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:00:52.046119 17943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:52.046352 17919 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:52.046490 17919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "2a49b75f562b47e78bce2dbadc470d09"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "effb16b9e8dcac1116dedd824f7bdd3b"
server_key_iv: "a2e6fc880447bf5a9f96040ab6c1b6b1"
server_key_version: "encryptionkey@0"
I20260812 04:00:52.047035 17919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:52.075290 17919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:52.077682 17919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:52.077868 17919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:52.078195 17919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:52.078533 17919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:52.078593 17919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:52.078634 17919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:52.078678 17919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:52.085207 17919 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:41829
I20260812 04:00:52.085256 18067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:41829 every 8 connection(s)
I20260812 04:00:52.085682 17919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:00:52.088230 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 17919
I20260812 04:00:52.088569 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:44585
--local_ip_for_outbound_sockets=127.31.43.195
--tserver_master_addrs=127.31.43.254:33795
--webserver_port=41503
--webserver_interface=127.31.43.195
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 04:00:52 dist-test-slave-8tdl krb5kdc[17047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507252, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:52.096113 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.086606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:39247 (local address 127.31.43.254:33795)
0812 04:00:52.086706 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:52.086709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:52.086952 (+ 243us) server_negotiation.cc:407] Connection header received
0812 04:00:52.087550 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:52.087551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:52.087568 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:52.087630 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:52.088733 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.089257 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.090257 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.090496 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.090583 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.092937 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:52.092939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:52.092939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:52.092948 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:52.094479 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:52.095038 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:52.095044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:52.095045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:52.095103 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:52.095376 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:52.095377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:52.095377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:52.095508 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:52.095584 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:52.095908 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 04:00:52.096012 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 04:00:52.096325 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.086597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:39247)
0812 04:00:52.086815 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:52.086857 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:52.087425 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:52.087686 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:52.087690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:52.088317 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:52.088566 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:52.088572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.089355 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.089356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:52.090158 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:52.090161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.090592 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.090593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:52.090742 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.091333 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:52.091341 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:52.092830 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:52.094591 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:52.094600 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:52.094603 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:52.094943 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:52.095178 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:52.095179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:52.095179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:52.095300 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:52.095587 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:52.095595 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:52.095763 (+ 168us) client_negotiation.cc:769] Sending connection context
0812 04:00:52.095881 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 04:00:52.096071 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":124,"threads_started":1}
I20260812 04:00:52.096870 18068 heartbeater.cc:347] Connected to a master server at 127.31.43.254:33795
I20260812 04:00:52.097002 18068 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:52.097247 18068 heartbeater.cc:511] Master 127.31.43.254:33795 requested a full tablet report, sending...
I20260812 04:00:52.097829 17703 ts_manager.cc:194] Registered new tserver with Master: 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829)
I20260812 04:00:52.098232 17703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:39247
W20260812 04:00:52.178404 18071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:00:52.178659 18071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:00:52.178720 18071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:00:52.179942 18071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:00:52.180014 18071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:00:52.180063 18071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:00:52.181825 18071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:44585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=41503
--tserver_master_addrs=127.31.43.254:33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:00:52.182217 18071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:00:52.182474 18071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:00:52.188067 18076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:52.188341 18071 server_base.cc:1034] running on GCE node
W20260812 04:00:52.188081 18080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:00:52.188097 18077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:00:52.188691 18071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:00:52.188987 18071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:00:52.190168 18071 hybrid_clock.cc:648] HybridClock initialized: now 1786507252190152 us; error 53 us; skew 500 ppm
Aug 12 04:00:52 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507252, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:00:52.193104 18071 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 04:00:52.193653 18071 webserver.cc:460] Webserver started at http://127.31.43.195:41503/ using document root <none> and password file <none>
I20260812 04:00:52.193923 18071 fs_manager.cc:359] Metadata directory not provided
I20260812 04:00:52.194015 18071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:00:52.195772 18071 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:52.196600 18087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:52.196776 18071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:00:52.196866 18071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "9ce616a372a54fe98ed432a263a734b4"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "93b698ce9a7ef15433a31270be5e097a"
server_key_iv: "b44ea206e028e0960cc1b6f17acbeac9"
server_key_version: "encryptionkey@0"
I20260812 04:00:52.197250 18071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:00:52.211476 18071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:00:52.213774 18071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:00:52.214025 18071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:00:52.214303 18071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:00:52.214661 18071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:00:52.214780 18071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:52.214839 18071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:00:52.214879 18071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:00:52.221817 18071 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:44585
I20260812 04:00:52.221896 18214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:44585 every 8 connection(s)
I20260812 04:00:52.222321 18071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 04:00:52.225553 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 18071
Aug 12 04:00:52 dist-test-slave-8tdl krb5kdc[17047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507252, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:00:52.234222 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.223272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:45655 (local address 127.31.43.254:33795)
0812 04:00:52.223368 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:52.223371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:52.223516 (+ 145us) server_negotiation.cc:407] Connection header received
0812 04:00:52.224694 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:52.224696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:52.224709 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:52.224775 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:52.225678 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.226541 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.227873 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.228255 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.228340 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.230833 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:52.230835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:52.230835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:52.230844 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:52.232526 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:52.232991 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:52.232992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:52.232993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:52.233063 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:52.233396 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:52.233397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:52.233397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:52.233550 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:52.233649 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:52.234006 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 04:00:52.234111 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 04:00:52.234500 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.226009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.31.43.254:33795)
0812 04:00:52.226114 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:52.226116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:52.226249 (+ 133us) server_negotiation.cc:407] Connection header received
0812 04:00:52.226314 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:52.226315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:52.226325 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:52.226388 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:52.227185 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.227920 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.228939 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.229251 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.229331 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.231537 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:52.231539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:52.231539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:52.231546 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:52.233179 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:52.233620 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:52.233621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:52.233622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:52.233685 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:52.233918 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:52.233919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:52.233919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:52.234056 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:52.234241 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:52.234318 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 04:00:52.234417 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":52,"threads_started":1}
I20260812 04:00:52.234974 18217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.223191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:45655)
0812 04:00:52.223368 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:52.223421 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:52.224571 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:52.224900 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:52.224904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:52.225222 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:00:52.225536 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:52.225544 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.226646 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.226646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:52.227784 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:52.227786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.228357 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.228359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:52.228475 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.229119 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:00:52.229129 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:00:52.230740 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:00:52.232627 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:52.232635 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:52.232638 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:52.232896 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:52.233141 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:00:52.233142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:00:52.233143 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:00:52.233317 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:00:52.233682 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:00:52.233690 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:00:52.233848 (+ 158us) client_negotiation.cc:769] Sending connection context
0812 04:00:52.233981 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 04:00:52.234170 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260812 04:00:52.235626 18215 heartbeater.cc:347] Connected to a master server at 127.31.43.254:33795
I20260812 04:00:52.235731 18215 heartbeater.cc:464] Registering TS with master...
I20260812 04:00:52.235889 18215 heartbeater.cc:511] Master 127.31.43.254:33795 requested a full tablet report, sending...
I20260812 04:00:52.236307 17702 ts_manager.cc:194] Registered new tserver with Master: 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585)
I20260812 04:00:52.236656 17702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:45655
I20260812 04:00:52.246781 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 04:00:52.250311 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.247375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41770 (local address 127.31.43.254:33795)
0812 04:00:52.247456 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:52.247458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:52.247469 (+ 11us) server_negotiation.cc:407] Connection header received
0812 04:00:52.247592 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:52.247593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:52.247637 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:52.247728 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:00:52.247898 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.248504 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.249380 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.249616 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.249706 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.249909 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:00:52.249989 (+ 80us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 04:00:52.250219 (+ 230us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41770: token expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 04:00:52.250378 18219 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41770: token expired
W20260812 04:00:52.250577 17626 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.43.254:33795)
I20260812 04:00:52.257800 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.250939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41786 (local address 127.31.43.254:33795)
0812 04:00:52.251008 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:52.251010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:52.251085 (+ 75us) server_negotiation.cc:407] Connection header received
0812 04:00:52.251174 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:52.251176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:52.251187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:52.251267 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:00:52.251845 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.252362 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.253170 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.253380 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.253483 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.254923 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:00:52.254925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:00:52.254925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:00:52.254938 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:00:52.256511 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:52.256922 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:52.256923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:52.256924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:52.256990 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:00:52.257219 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:00:52.257220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:00:52.257221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:00:52.257340 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:00:52.257460 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:52.257577 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 04:00:52.257674 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 04:00:52.259341 17702 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 04:00:52.261214 17702 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 04:00:52.263808 17821 tablet_service.cc:1467] Processing CreateTablet for tablet 0c72a47f069d49749fde740d163afb6f (DEFAULT_TABLE table=test-table [id=c0979228975b4c6d83343cc5d5e419c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:52.264055 17821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c72a47f069d49749fde740d163afb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:52.265765 17794 tablet_bootstrap.cc:492] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e: Bootstrap starting.
I20260812 04:00:52.266160 17794 tablet_bootstrap.cc:654] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:52.266826 17794 tablet_bootstrap.cc:492] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e: No bootstrap required, opened a new log
I20260812 04:00:52.266884 17794 ts_tablet_manager.cc:1397] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 04:00:52.267282 17794 raft_consensus.cc:348] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:00:52.267371 17794 raft_consensus.cc:374] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:52.267424 17794 raft_consensus.cc:729] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2e85f133ae41ceb4ec6549f9e8701e, State: Initialized, Role: FOLLOWER
I20260812 04:00:52.267474 17794 consensus_queue.cc:260] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:00:52.267577 17794 ts_tablet_manager.cc:1428] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 04:00:52.269078 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.263734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43636 (local address 127.31.43.194:41829)
0812 04:00:52.264005 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:52.264010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:52.264020 (+ 10us) server_negotiation.cc:407] Connection header received
0812 04:00:52.264061 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:52.264064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:52.264080 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:52.264359 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:52.264455 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.265545 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.267629 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.268603 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.268710 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.268769 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:52.268939 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 04:00:52.268984 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":87,"threads_started":1}
I20260812 04:00:52.269062 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.263490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:41829 (local address 127.0.0.1:43636)
0812 04:00:52.263738 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:52.263748 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:52.263802 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:52.264223 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:52.264224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:52.264226 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:52.264282 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:52.264284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.265641 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.265641 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:52.267457 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:52.267460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.268720 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.268721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:52.268809 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.268835 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 04:00:52.268900 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 04:00:52.268959 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":88,"threads_started":1}
I20260812 04:00:52.269749 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.263474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:44585 (local address 127.0.0.1:48766)
0812 04:00:52.263536 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:52.263550 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:52.263593 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:52.264156 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:52.264156 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:52.264159 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:52.264244 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:52.264246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.265438 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.265440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:52.266945 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:52.266949 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.269444 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.269447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:52.269548 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.269554 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 04:00:52.269617 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 04:00:52.269673 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":3}
I20260812 04:00:52.269779 18222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.263680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.31.43.195:44585)
0812 04:00:52.263959 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:52.263967 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:52.263980 (+ 13us) server_negotiation.cc:407] Connection header received
0812 04:00:52.264030 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:52.264033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:52.264050 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:52.264149 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:52.264397 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.265329 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.267376 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.268485 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.268569 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.268641 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:52.269639 (+ 998us) server_negotiation.cc:299] Negotiation successful
0812 04:00:52.269689 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":130,"threads_started":1}
I20260812 04:00:52.269965 17999 tablet_service.cc:1467] Processing CreateTablet for tablet 0c72a47f069d49749fde740d163afb6f (DEFAULT_TABLE table=test-table [id=c0979228975b4c6d83343cc5d5e419c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:52.270386 17999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c72a47f069d49749fde740d163afb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:52.271132 18142 tablet_service.cc:1467] Processing CreateTablet for tablet 0c72a47f069d49749fde740d163afb6f (DEFAULT_TABLE table=test-table [id=c0979228975b4c6d83343cc5d5e419c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:00:52.271452 18142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c72a47f069d49749fde740d163afb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:00:52.273607 18225 tablet_bootstrap.cc:492] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09: Bootstrap starting.
I20260812 04:00:52.274271 18225 tablet_bootstrap.cc:654] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:52.274760 18226 tablet_bootstrap.cc:492] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4: Bootstrap starting.
I20260812 04:00:52.274972 18225 log.cc:824] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:52.275441 18226 tablet_bootstrap.cc:654] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4: Neither blocks nor log segments found. Creating new log.
I20260812 04:00:52.276000 18225 tablet_bootstrap.cc:492] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09: No bootstrap required, opened a new log
I20260812 04:00:52.276038 18226 log.cc:824] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4: Log is configured to *not* fsync() on all Append() calls
I20260812 04:00:52.276077 18225 ts_tablet_manager.cc:1397] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 04:00:52.276831 18226 tablet_bootstrap.cc:492] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4: No bootstrap required, opened a new log
I20260812 04:00:52.276899 18226 ts_tablet_manager.cc:1397] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 04:00:52.278110 18225 raft_consensus.cc:348] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:00:52.278323 18225 raft_consensus.cc:374] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:52.278362 18225 raft_consensus.cc:729] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a49b75f562b47e78bce2dbadc470d09, State: Initialized, Role: FOLLOWER
I20260812 04:00:52.278481 18225 consensus_queue.cc:260] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:00:52.278821 18225 ts_tablet_manager.cc:1428] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 04:00:52.278872 18068 heartbeater.cc:503] Master 127.31.43.254:33795 was elected leader, sending a full tablet report...
I20260812 04:00:52.278920 18226 raft_consensus.cc:348] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:00:52.279071 18226 raft_consensus.cc:374] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:00:52.279094 18226 raft_consensus.cc:729] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ce616a372a54fe98ed432a263a734b4, State: Initialized, Role: FOLLOWER
I20260812 04:00:52.279204 18226 consensus_queue.cc:260] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:00:52.279492 18226 ts_tablet_manager.cc:1428] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 04:00:52.279610 18215 heartbeater.cc:503] Master 127.31.43.254:33795 was elected leader, sending a full tablet report...
W20260812 04:00:52.336387 18069 tablet.cc:2369] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:00:52.371301 17920 raft_consensus.cc:482] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:00:52.371464 17920 raft_consensus.cc:504] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:00:52.371820 17920 leader_election.cc:288] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585), 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829)
I20260812 04:00:52.375535 17916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.372061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:41829 (local address 127.31.43.193:50431)
0812 04:00:52.372146 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:52.372161 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:52.372259 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:52.372502 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:52.372503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:52.372505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:52.372598 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:52.372601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.373517 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.373518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:52.374492 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:52.374495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.375208 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.375209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:52.375304 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.375342 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 04:00:52.375405 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 04:00:52.375463 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":1}
I20260812 04:00:52.375567 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.372275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:50431 (local address 127.31.43.194:41829)
0812 04:00:52.372355 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:52.372357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:52.372369 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:00:52.372403 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:52.372405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:52.372416 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:52.372480 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:52.372684 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.373424 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.374582 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.375110 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.375174 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.375182 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:52.375429 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 04:00:52.375481 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 04:00:52.375795 18222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.372150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:48329 (local address 127.31.43.195:44585)
0812 04:00:52.372215 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:52.372218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:52.372372 (+ 154us) server_negotiation.cc:407] Connection header received
0812 04:00:52.372449 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:52.372496 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:52.372508 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:52.372571 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:00:52.372790 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.373429 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.374764 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.375310 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.375401 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.375412 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:52.375667 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 04:00:52.375724 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 04:00:52.376044 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.372061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:44585 (local address 127.31.43.193:48329)
0812 04:00:52.372261 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:52.372272 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:00:52.372372 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:00:52.372594 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:00:52.372595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:00:52.372597 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:00:52.372706 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:52.372708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.373527 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.373528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:52.374633 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:00:52.374637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.375461 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.375463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:00:52.375550 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.375554 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:00:52.375894 (+ 340us) client_negotiation.cc:240] Negotiation successful
0812 04:00:52.375949 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":50,"thread_start_us":70,"threads_started":1}
I20260812 04:00:52.375967 18020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c72a47f069d49749fde740d163afb6f" candidate_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a49b75f562b47e78bce2dbadc470d09" is_pre_election: true
I20260812 04:00:52.376214 18020 raft_consensus.cc:2454] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c2e85f133ae41ceb4ec6549f9e8701e in term 0.
I20260812 04:00:52.376446 18163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c72a47f069d49749fde740d163afb6f" candidate_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ce616a372a54fe98ed432a263a734b4" is_pre_election: true
I20260812 04:00:52.376500 17786 leader_election.cc:302] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a49b75f562b47e78bce2dbadc470d09, 7c2e85f133ae41ceb4ec6549f9e8701e; no voters:
I20260812 04:00:52.376622 18163 raft_consensus.cc:2454] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c2e85f133ae41ceb4ec6549f9e8701e in term 0.
I20260812 04:00:52.376660 17920 raft_consensus.cc:2781] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 04:00:52.376737 17920 raft_consensus.cc:482] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 04:00:52.376773 17920 raft_consensus.cc:3037] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:52.377569 17920 raft_consensus.cc:504] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:00:52.377749 17920 leader_election.cc:288] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 1 election: Requested vote from peers 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585), 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829)
I20260812 04:00:52.377924 18020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c72a47f069d49749fde740d163afb6f" candidate_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a49b75f562b47e78bce2dbadc470d09"
I20260812 04:00:52.377938 18163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c72a47f069d49749fde740d163afb6f" candidate_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ce616a372a54fe98ed432a263a734b4"
I20260812 04:00:52.378048 18020 raft_consensus.cc:3037] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:52.378048 18163 raft_consensus.cc:3037] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:00:52.379041 18020 raft_consensus.cc:2454] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c2e85f133ae41ceb4ec6549f9e8701e in term 1.
I20260812 04:00:52.379055 18163 raft_consensus.cc:2454] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c2e85f133ae41ceb4ec6549f9e8701e in term 1.
I20260812 04:00:52.379240 17786 leader_election.cc:302] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a49b75f562b47e78bce2dbadc470d09, 7c2e85f133ae41ceb4ec6549f9e8701e; no voters:
I20260812 04:00:52.379381 17920 raft_consensus.cc:2781] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:00:52.379562 17920 raft_consensus.cc:686] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 LEADER]: Becoming Leader. State: Replica: 7c2e85f133ae41ceb4ec6549f9e8701e, State: Running, Role: LEADER
I20260812 04:00:52.379629 17920 consensus_queue.cc:237] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:00:52.380600 17702 catalog_manager.cc:5495] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c2e85f133ae41ceb4ec6549f9e8701e (127.31.43.193). New cstate: current_term: 1 leader_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } health_report { overall_health: HEALTHY } } }
I20260812 04:00:52.386655 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.384190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51684 (local address 127.31.43.193:45939)
0812 04:00:52.384256 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 04:00:52.384259 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:00:52.384270 (+ 11us) server_negotiation.cc:407] Connection header received
0812 04:00:52.384321 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:00:52.384322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:00:52.384344 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:00:52.384419 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:00:52.384561 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.385060 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.385819 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:00:52.386037 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:00:52.386117 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:00:52.386300 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:00:52.386405 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:00:52.386486 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 04:00:52.386535 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 04:00:52.386560 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 04:00:52.388540 18020 raft_consensus.cc:1261] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09 [term 1 FOLLOWER]: Refusing update from remote peer 7c2e85f133ae41ceb4ec6549f9e8701e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:00:52.388540 18163 raft_consensus.cc:1261] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4 [term 1 FOLLOWER]: Refusing update from remote peer 7c2e85f133ae41ceb4ec6549f9e8701e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:00:52.389161 17920 consensus_queue.cc:1035] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:00:52.389345 18234 consensus_queue.cc:1035] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:00:52.392035 18237 mvcc.cc:205] Tried to move back new op lower bound from 7317533705780330496 to 7317533705747763200. Current Snapshot: MvccSnapshot[applied={T|T < 7317533705780330496}]
I20260812 04:00:52.392935 18238 mvcc.cc:205] Tried to move back new op lower bound from 7317533705780330496 to 7317533705747763200. Current Snapshot: MvccSnapshot[applied={T|T < 7317533705780330496}]
I20260812 04:00:52.393401 18240 mvcc.cc:205] Tried to move back new op lower bound from 7317533705780330496 to 7317533705747763200. Current Snapshot: MvccSnapshot[applied={T|T < 7317533705780330496}]
I20260812 04:00:52.394835 17703 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:00:52.394953 17703 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:00:52.395810 17703 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Sending DeleteTablet for 3 replicas of tablet 0c72a47f069d49749fde740d163afb6f
I20260812 04:00:52.396195 18142 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c72a47f069d49749fde740d163afb6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:52 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:48766
I20260812 04:00:52.396299 17999 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c72a47f069d49749fde740d163afb6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:52 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:43636
I20260812 04:00:52.396399 17821 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c72a47f069d49749fde740d163afb6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:00:52 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:51668
I20260812 04:00:52.396503 17925 tablet_replica.cc:330] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e: stopping tablet replica
I20260812 04:00:52.396580 17925 raft_consensus.cc:2229] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 LEADER]: Raft consensus shutting down.
I20260812 04:00:52.396617 18247 tablet_replica.cc:330] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4: stopping tablet replica
I20260812 04:00:52.396634 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 17745
I20260812 04:00:52.396675 17925 raft_consensus.cc:2258] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:00:52.400422 18248 tablet_replica.cc:330] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09: stopping tablet replica
I20260812 04:00:52.400583 18248 raft_consensus.cc:2229] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:00:52.400719 18248 raft_consensus.cc:2258] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:00:52.401268 18248 ts_tablet_manager.cc:1905] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:00:52.403617 18248 ts_tablet_manager.cc:1918] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:00:52.403714 18248 log.cc:1196] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/0c72a47f069d49749fde740d163afb6f
I20260812 04:00:52.396713 18247 raft_consensus.cc:2229] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:00:52.404060 18248 ts_tablet_manager.cc:1939] T 0c72a47f069d49749fde740d163afb6f P 2a49b75f562b47e78bce2dbadc470d09: Deleting consensus metadata
I20260812 04:00:52.404247 18247 raft_consensus.cc:2258] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:00:52.404562 17687 catalog_manager.cc:4841] TS 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829): tablet 0c72a47f069d49749fde740d163afb6f (table test-table [id=c0979228975b4c6d83343cc5d5e419c5]) successfully deleted
I20260812 04:00:52.404821 18247 ts_tablet_manager.cc:1905] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:00:52.406973 18247 ts_tablet_manager.cc:1918] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:00:52.407060 18247 log.cc:1196] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/0c72a47f069d49749fde740d163afb6f
I20260812 04:00:52.407341 18247 ts_tablet_manager.cc:1939] T 0c72a47f069d49749fde740d163afb6f P 9ce616a372a54fe98ed432a263a734b4: Deleting consensus metadata
I20260812 04:00:52.407842 17690 catalog_manager.cc:4841] TS 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585): tablet 0c72a47f069d49749fde740d163afb6f (table test-table [id=c0979228975b4c6d83343cc5d5e419c5]) successfully deleted
W20260812 04:00:52.409175 17689 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:45939 (error 108)
I20260812 04:00:52.409636 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 17919
I20260812 04:00:52.409808 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:00:52.409647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45939 (local address 127.0.0.1:51686)
0812 04:00:52.409720 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 04:00:52.409767 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:45939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 04:00:52.412969 17689 catalog_manager.cc:4584] TS 7c2e85f133ae41ceb4ec6549f9e8701e (127.31.43.193:45939): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0c72a47f069d49749fde740d163afb6f: Network error: Client connection negotiation failed: client connection to 127.31.43.193:45939: connect: Connection refused (error 111)
I20260812 04:00:52.419028 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 18071
I20260812 04:00:52.428195 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 17670
I20260812 04:01:12.437233 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:33795
--webserver_interface=127.31.43.254
--webserver_port=38639
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:33795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:12.524904 18250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:12.525143 18250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:12.525168 18250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:12.526343 18250 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 04:01:12.526382 18250 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 04:01:12.526396 18250 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 04:01:12.526408 18250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:12.526422 18250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:12.526435 18250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:12.526448 18250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:12.526459 18250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:12.527976 18250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:33795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:33795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=38639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:12.528210 18250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:12.528395 18250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:12.534078 18256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:12.534219 18250 server_base.cc:1034] running on GCE node
W20260812 04:01:12.534112 18258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:12.534307 18255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:12.534623 18250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:12.534988 18250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:12.536166 18250 hybrid_clock.cc:648] HybridClock initialized: now 1786507272536185 us; error 79 us; skew 500 ppm
Aug 12 04:01:12 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507272, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:12.543913 18250 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:01:12.544409 18250 webserver.cc:460] Webserver started at http://127.31.43.254:38639/ using document root <none> and password file <none>
I20260812 04:01:12.544668 18250 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:12.544744 18250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:12.546236 18250 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 04:01:12.547060 18265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:12.547242 18250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:12.547354 18250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "512ac5d959fb4f628c94a91614366ead"
format_stamp: "Formatted at 2026-08-12 04:00:30 on dist-test-slave-8tdl"
server_key: "47b3ba48346e1899ee04a8aabd6c21a8"
server_key_iv: "5884426705c6f9d3b8957716800c331e"
server_key_version: "encryptionkey@0"
I20260812 04:01:12.547755 18250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:12.567298 18250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:12.570295 18250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:12.570472 18250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:12.575223 18250 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:33795
I20260812 04:01:12.575255 18325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:33795 every 8 connection(s)
I20260812 04:01:12.575829 18250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:12.577224 18326 sys_catalog.cc:263] Verifying existing consensus state
I20260812 04:01:12.577852 18326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Bootstrap starting.
I20260812 04:01:12.584290 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 18250
I20260812 04:01:12.584626 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:45939
--local_ip_for_outbound_sockets=127.31.43.193
--tserver_master_addrs=127.31.43.254:33795
--webserver_port=45227
--webserver_interface=127.31.43.193
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:01:12.587894 18326 log.cc:824] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:12.590293 18326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 04:01:12.590643 18326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Bootstrap complete.
I20260812 04:01:12.593086 18326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } }
I20260812 04:01:12.593327 18326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512ac5d959fb4f628c94a91614366ead, State: Initialized, Role: FOLLOWER
I20260812 04:01:12.593514 18326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } }
I20260812 04:01:12.593581 18326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:12.593647 18326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:12.593698 18326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 2 FOLLOWER]: Advancing to term 3
I20260812 04:01:12.594827 18326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } }
I20260812 04:01:12.594939 18326 leader_election.cc:302] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 512ac5d959fb4f628c94a91614366ead; no voters:
I20260812 04:01:12.595201 18326 leader_election.cc:288] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 04:01:12.595243 18332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 3 FOLLOWER]: Leader election won for term 3
I20260812 04:01:12.595378 18332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [term 3 LEADER]: Becoming Leader. State: Replica: 512ac5d959fb4f628c94a91614366ead, State: Running, Role: LEADER
I20260812 04:01:12.595487 18326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:12.595466 18332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } }
I20260812 04:01:12.595902 18334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "512ac5d959fb4f628c94a91614366ead" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } } }
I20260812 04:01:12.595909 18333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: SysCatalogTable state changed. Reason: New leader 512ac5d959fb4f628c94a91614366ead. Latest consensus state: current_term: 3 leader_uuid: "512ac5d959fb4f628c94a91614366ead" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512ac5d959fb4f628c94a91614366ead" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33795 } } }
I20260812 04:01:12.596107 18334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:12.596127 18333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:12.596508 18341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:12.597486 18341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:12.597777 18341 catalog_manager.cc:1259] Loaded cluster ID: 3390aca7ed1b48b495a5ff5ac2d390e3
I20260812 04:01:12.597855 18341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:12.599105 18341 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:12.599279 18341 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Loaded TSK: 0
I20260812 04:01:12.599323 18341 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 512ac5d959fb4f628c94a91614366ead: Loaded TSK: 1
F20260812 04:01:12.599575 18341 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f44846db9ad google::LogMessage::Fail() at ??:0
@ 0x7f44846de88a google::LogMessage::SendToLog() at ??:0
@ 0x7f44846db4de google::LogMessage::Flush() at ??:0
@ 0x7f44846dca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4486e48104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f4486e45d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f448c6351cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f448c63548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f448c635594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f448c5fbc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f448c605ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f44865a90d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f448659e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f44874026db start_thread at ??:0
@ 0x7f44831d961f clone at ??:0
W20260812 04:01:12.691381 18330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:12.691630 18330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:12.691670 18330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:12.692836 18330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:12.692905 18330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:12.692951 18330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:01:12.694545 18330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:45939
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=45227
--tserver_master_addrs=127.31.43.254:33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:12.694859 18330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:12.695205 18330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:12.700520 18375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:12.700531 18378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:12.700558 18376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:12.700912 18330 server_base.cc:1034] running on GCE node
I20260812 04:01:12.701144 18330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:12.701443 18330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:12.702651 18330 hybrid_clock.cc:648] HybridClock initialized: now 1786507272702635 us; error 39 us; skew 500 ppm
Aug 12 04:01:12 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507272, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:12.705888 18330 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 04:01:12.706508 18330 webserver.cc:460] Webserver started at http://127.31.43.193:45227/ using document root <none> and password file <none>
I20260812 04:01:12.706822 18330 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:12.706900 18330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:12.708804 18330 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 04:01:12.709654 18385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:12.709858 18330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:12.709970 18330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "7c2e85f133ae41ceb4ec6549f9e8701e"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "a666a2cee3e1846234ac27aab8282215"
server_key_iv: "fa07fd0d109afc2c02942528189de6ff"
server_key_version: "encryptionkey@0"
I20260812 04:01:12.710356 18330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:12.724123 18330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:12.726643 18330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:12.726881 18330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:12.727145 18330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:12.727649 18392 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 04:01:12.729755 18330 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 04:01:12.729835 18330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 04:01:12.729880 18330 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 04:01:12.730592 18330 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 04:01:12.730655 18330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 04:01:12.730782 18392 tablet_bootstrap.cc:492] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e: Bootstrap starting.
I20260812 04:01:12.736809 18330 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:45939
I20260812 04:01:12.737355 18330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 04:01:12.737861 18513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:45939 every 8 connection(s)
I20260812 04:01:12.741602 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 18330
I20260812 04:01:12.741937 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:41829
--local_ip_for_outbound_sockets=127.31.43.194
--tserver_master_addrs=127.31.43.254:33795
--webserver_port=34301
--webserver_interface=127.31.43.194
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:01:12.742664 18392 log.cc:824] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:12.745687 18392 tablet_bootstrap.cc:492] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 04:01:12.746048 18392 tablet_bootstrap.cc:492] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e: Bootstrap complete.
I20260812 04:01:12.747423 18392 ts_tablet_manager.cc:1397] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.006s
I20260812 04:01:12.748970 18392 raft_consensus.cc:348] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:01:12.749321 18392 raft_consensus.cc:729] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2e85f133ae41ceb4ec6549f9e8701e, State: Initialized, Role: FOLLOWER
I20260812 04:01:12.749495 18392 consensus_queue.cc:260] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:01:12.749799 18392 ts_tablet_manager.cc:1428] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260812 04:01:12.816565 18516 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:01:12.738503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:43237)
0812 04:01:12.738746 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:12.738792 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:12.739281 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:12.816362 (+ 77081us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":111,"threads_started":1}
W20260812 04:01:12.816817 18514 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:33795 (0 consecutive failures): Network error: Failed to ping master at 127.31.43.254:33795: Client connection negotiation failed: client connection to 127.31.43.254:33795: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 04:01:12.817175 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:12.817034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:39181)
0812 04:01:12.817111 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:12.817155 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 04:01:12.817584 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:12.817491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:36879)
0812 04:01:12.817542 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:12.817566 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 04:01:12.817728 18514 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 04:01:12.833169 18519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:12.833423 18519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:12.833464 18519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:12.834651 18519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:12.834744 18519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:12.834792 18519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:01:12.836313 18519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:41829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=34301
--tserver_master_addrs=127.31.43.254:33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:12.836614 18519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:12.836856 18519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:12.842453 18530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:12.842447 18528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:12.842453 18527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:12.842855 18519 server_base.cc:1034] running on GCE node
I20260812 04:01:12.843072 18519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:12.843320 18519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:12.844485 18519 hybrid_clock.cc:648] HybridClock initialized: now 1786507272844478 us; error 37 us; skew 500 ppm
Aug 12 04:01:12 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507272, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:12.847222 18519 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 04:01:12.847739 18519 webserver.cc:460] Webserver started at http://127.31.43.194:34301/ using document root <none> and password file <none>
I20260812 04:01:12.847977 18519 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:12.848028 18519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:12.849449 18519 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:12.850106 18538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:12.850283 18519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:12.850394 18519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "2a49b75f562b47e78bce2dbadc470d09"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "effb16b9e8dcac1116dedd824f7bdd3b"
server_key_iv: "a2e6fc880447bf5a9f96040ab6c1b6b1"
server_key_version: "encryptionkey@0"
I20260812 04:01:12.850827 18519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:12.861671 18519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:12.864135 18519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:12.864302 18519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:12.864573 18519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:12.864910 18519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:12.864948 18519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:12.864971 18519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:12.864984 18519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:12.871398 18519 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:41829
I20260812 04:01:12.871479 18663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:41829 every 8 connection(s)
I20260812 04:01:12.871855 18519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:01:12.872975 18666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:12.872595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:55631)
0812 04:01:12.872757 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:12.872831 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
W20260812 04:01:12.873410 18664 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:33795 (0 consecutive failures): Network error: Failed to ping master at 127.31.43.254:33795: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
I20260812 04:01:12.873718 18666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:12.873613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:54235)
0812 04:01:12.873665 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:12.873696 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 04:01:12.874025 18666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:12.873952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:45069)
0812 04:01:12.873989 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:12.874011 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 04:01:12.874150 18664 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 04:01:12.881039 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 18519
I20260812 04:01:12.881357 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:44585
--local_ip_for_outbound_sockets=127.31.43.195
--tserver_master_addrs=127.31.43.254:33795
--webserver_port=41503
--webserver_interface=127.31.43.195
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:12.968582 18667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:12.968849 18667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:12.968891 18667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:12.970063 18667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:12.970132 18667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:12.970180 18667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:01:12.971717 18667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:44585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=41503
--tserver_master_addrs=127.31.43.254:33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:12.971990 18667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:12.972255 18667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:12.977963 18673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:12.977962 18676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:12.978109 18674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:12.978437 18667 server_base.cc:1034] running on GCE node
I20260812 04:01:12.978633 18667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:12.978914 18667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:12.980111 18667 hybrid_clock.cc:648] HybridClock initialized: now 1786507272980099 us; error 34 us; skew 500 ppm
Aug 12 04:01:12 dist-test-slave-8tdl krb5kdc[17047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507272, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:12.982822 18667 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 04:01:12.983335 18667 webserver.cc:460] Webserver started at http://127.31.43.195:41503/ using document root <none> and password file <none>
I20260812 04:01:12.983529 18667 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:12.983572 18667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:12.985069 18667 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:12.985678 18684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:12.985857 18667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:12.985965 18667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "9ce616a372a54fe98ed432a263a734b4"
format_stamp: "Formatted at 2026-08-12 04:00:31 on dist-test-slave-8tdl"
server_key: "93b698ce9a7ef15433a31270be5e097a"
server_key_iv: "b44ea206e028e0960cc1b6f17acbeac9"
server_key_version: "encryptionkey@0"
I20260812 04:01:12.986341 18667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:12.995869 18667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:12.997999 18667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:12.998176 18667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:12.998397 18667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:12.998754 18667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:12.998816 18667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:12.998858 18667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:12.998884 18667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:13.004853 18667 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:44585
I20260812 04:01:13.004943 18807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:44585 every 8 connection(s)
I20260812 04:01:13.005336 18667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 04:01:13.006613 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:13.006126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:46581)
0812 04:01:13.006334 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:13.006419 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":85,"threads_started":1}
W20260812 04:01:13.007135 18808 heartbeater.cc:644] Failed to heartbeat to 127.31.43.254:33795 (0 consecutive failures): Network error: Failed to ping master at 127.31.43.254:33795: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
I20260812 04:01:13.007460 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:13.007343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:51101)
0812 04:01:13.007392 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:13.007422 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 04:01:13.007794 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:13.007682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:48829)
0812 04:01:13.007734 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:13.007766 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 04:01:13.007803 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 18667
W20260812 04:01:13.007972 18808 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 04:01:13.818477 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:13.818185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:35261)
0812 04:01:13.818355 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:13.818425 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260812 04:01:13.874840 18814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:13.874541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:57713)
0812 04:01:13.874728 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:13.874767 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
I20260812 04:01:13.964632 18815 raft_consensus.cc:482] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:01:13.964774 18815 raft_consensus.cc:504] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:01:13.965116 18815 leader_election.cc:288] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585), 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829)
Aug 12 04:01:13 dist-test-slave-8tdl krb5kdc[17047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507272, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.194@KRBTEST.COM
Aug 12 04:01:13 dist-test-slave-8tdl krb5kdc[17047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507272, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.195@KRBTEST.COM
I20260812 04:01:13.977222 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:13.965341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:41829 (local address 127.31.43.193:47341)
0812 04:01:13.965547 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:13.965560 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:13.965655 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:13.966524 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:13.966525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:13.967003 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:13.967198 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:13.967204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:13.968432 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:13.968433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:13.969256 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:13.969259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:13.970124 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:13.970125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:13.970265 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:13.970898 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:01:13.970908 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:13.972811 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:13.975199 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:13.975208 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:13.975211 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:13.975552 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:13.975853 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:13.975854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:13.975854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:13.975998 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:13.976570 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:13.976600 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:13.976767 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 04:01:13.976875 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 04:01:13.977092 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":93,"thread_start_us":56,"threads_started":1}
I20260812 04:01:13.977324 18818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:13.965536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:47341 (local address 127.31.43.194:41829)
0812 04:01:13.965679 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:13.965695 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:13.965724 (+ 29us) server_negotiation.cc:407] Connection header received
0812 04:01:13.966215 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:13.966218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:13.966269 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:13.966405 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:13.967354 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:13.968319 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:13.969387 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:13.969970 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:13.970125 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:13.972960 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:13.972964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:13.972965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:13.972982 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:13.975106 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:13.975641 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:13.975645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:13.975645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:13.975777 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:13.976174 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:13.976175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:13.976176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:13.976486 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:13.976558 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:13.977022 (+ 464us) server_negotiation.cc:299] Negotiation successful
0812 04:01:13.977192 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":66,"threads_started":1}
I20260812 04:01:13.977380 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:13.965340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:44585 (local address 127.31.43.193:49057)
0812 04:01:13.965415 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:13.965430 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:13.965528 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:13.966513 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:13.966516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:13.967003 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:13.967198 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:13.967205 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:13.969529 (+ 2324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:13.969531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:13.970334 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:13.970336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:13.971125 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:13.971127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:13.971194 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:13.971649 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:01:13.971655 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:13.973139 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:13.975371 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:13.975372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:13.975372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:13.975705 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:13.976176 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:13.976177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:13.976177 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:13.976247 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:13.976758 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:13.976759 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:13.976804 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 04:01:13.976962 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 04:01:13.977194 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":109}
I20260812 04:01:13.977680 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:13.965459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:49057 (local address 127.31.43.195:44585)
0812 04:01:13.965630 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:13.965644 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:13.965673 (+ 29us) server_negotiation.cc:407] Connection header received
0812 04:01:13.966197 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:13.966200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:13.966262 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:13.966468 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:13.968457 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:13.969409 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:13.970430 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:13.971035 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:13.971116 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:13.973260 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:13.973265 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:13.973265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:13.973278 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:13.975270 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:13.975797 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:13.975799 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:13.975800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:13.975952 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:13.976323 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:13.976324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:13.976324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:13.976643 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:13.976735 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:13.977040 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 04:01:13.977197 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":61,"threads_started":1}
W20260812 04:01:13.978266 18386 leader_election.cc:341] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829): Not found: Tablet not found: 0c72a47f069d49749fde740d163afb6f
W20260812 04:01:13.978643 18389 leader_election.cc:341] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585): Not found: Tablet not found: 0c72a47f069d49749fde740d163afb6f
I20260812 04:01:13.978763 18389 leader_election.cc:302] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c2e85f133ae41ceb4ec6549f9e8701e; no voters: 2a49b75f562b47e78bce2dbadc470d09, 9ce616a372a54fe98ed432a263a734b4
I20260812 04:01:13.978950 18815 raft_consensus.cc:2726] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 04:01:14.008806 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:14.008433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:43217)
0812 04:01:14.008653 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:14.008743 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":94,"threads_started":1}
I20260812 04:01:14.819279 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:14.819030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:50743)
0812 04:01:14.819189 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:14.819229 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260812 04:01:14.875689 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:14.875440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:32965)
0812 04:01:14.875596 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:14.875640 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260812 04:01:15.009621 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:15.009402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:47013)
0812 04:01:15.009530 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:15.009573 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260812 04:01:15.820258 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:15.819981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:47209)
0812 04:01:15.820164 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:15.820208 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":95,"threads_started":1}
I20260812 04:01:15.876471 18824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:15.876230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:38489)
0812 04:01:15.876388 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:15.876425 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":72,"threads_started":1}
I20260812 04:01:15.963603 18826 raft_consensus.cc:482] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:01:15.963737 18826 raft_consensus.cc:504] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:01:15.963963 18826 leader_election.cc:288] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585), 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829)
W20260812 04:01:15.964425 18389 leader_election.cc:341] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585): Not found: Tablet not found: 0c72a47f069d49749fde740d163afb6f
W20260812 04:01:15.964529 18386 leader_election.cc:341] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829): Not found: Tablet not found: 0c72a47f069d49749fde740d163afb6f
I20260812 04:01:15.964589 18386 leader_election.cc:302] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c2e85f133ae41ceb4ec6549f9e8701e; no voters: 2a49b75f562b47e78bce2dbadc470d09, 9ce616a372a54fe98ed432a263a734b4
I20260812 04:01:15.964663 18826 raft_consensus.cc:2726] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 04:01:16.010437 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:16.010182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:54785)
0812 04:01:16.010345 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:16.010391 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260812 04:01:16.821148 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:16.820832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:52527)
0812 04:01:16.820986 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:16.821033 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260812 04:01:16.877400 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:16.877162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:39451)
0812 04:01:16.877312 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:16.877356 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260812 04:01:17.011237 18831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:17.010985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:58887)
0812 04:01:17.011142 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:17.011188 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260812 04:01:17.822036 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:17.821772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:58563)
0812 04:01:17.821947 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:17.821988 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":70,"threads_started":1}
I20260812 04:01:17.878232 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:17.877989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:56031)
0812 04:01:17.878146 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:17.878186 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260812 04:01:18.012043 18834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:18.011789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:40621)
0812 04:01:18.011942 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:18.011984 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260812 04:01:18.618970 18835 raft_consensus.cc:482] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:01:18.619095 18835 raft_consensus.cc:504] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:01:18.619277 18835 leader_election.cc:288] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585), 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829)
W20260812 04:01:18.619690 18389 leader_election.cc:341] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585): Not found: Tablet not found: 0c72a47f069d49749fde740d163afb6f
W20260812 04:01:18.619758 18386 leader_election.cc:341] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829): Not found: Tablet not found: 0c72a47f069d49749fde740d163afb6f
I20260812 04:01:18.619835 18386 leader_election.cc:302] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c2e85f133ae41ceb4ec6549f9e8701e; no voters: 2a49b75f562b47e78bce2dbadc470d09, 9ce616a372a54fe98ed432a263a734b4
I20260812 04:01:18.619915 18835 raft_consensus.cc:2726] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 04:01:18.822924 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:18.822619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:57211)
0812 04:01:18.822839 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:18.822873 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":156,"threads_started":1}
I20260812 04:01:18.879156 18837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:18.878837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:46877)
0812 04:01:18.879064 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:18.879106 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":146,"threads_started":1}
I20260812 04:01:19.012920 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:19.012584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:36273)
0812 04:01:19.012771 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:19.012838 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":113,"threads_started":1}
I20260812 04:01:19.823652 18839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:19.823410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:45661)
0812 04:01:19.823561 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:19.823604 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260812 04:01:19.879978 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:19.879714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:53257)
0812 04:01:19.879875 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:19.879919 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
I20260812 04:01:20.013779 18841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:20.013545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:52645)
0812 04:01:20.013686 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:20.013730 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":67,"threads_started":1}
I20260812 04:01:20.824566 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:20.824302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:47471)
0812 04:01:20.824465 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:20.824518 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":82,"threads_started":1}
I20260812 04:01:20.880769 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:20.880529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:47143)
0812 04:01:20.880678 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:20.880716 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260812 04:01:21.014600 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:21.014310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:50495)
0812 04:01:21.014500 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:21.014550 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260812 04:01:21.429615 18845 raft_consensus.cc:482] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:01:21.429741 18845 raft_consensus.cc:504] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:01:21.429970 18845 leader_election.cc:288] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585), 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829)
W20260812 04:01:21.430486 18386 leader_election.cc:341] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829): Not found: Tablet not found: 0c72a47f069d49749fde740d163afb6f
W20260812 04:01:21.430589 18389 leader_election.cc:341] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585): Not found: Tablet not found: 0c72a47f069d49749fde740d163afb6f
I20260812 04:01:21.430653 18389 leader_election.cc:302] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c2e85f133ae41ceb4ec6549f9e8701e; no voters: 2a49b75f562b47e78bce2dbadc470d09, 9ce616a372a54fe98ed432a263a734b4
I20260812 04:01:21.430768 18845 raft_consensus.cc:2726] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 04:01:21.825352 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:21.825086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:45173)
0812 04:01:21.825259 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:21.825301 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260812 04:01:21.881560 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:21.881280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:51701)
0812 04:01:21.881457 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:21.881504 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":85,"threads_started":1}
I20260812 04:01:22.015502 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:22.015206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:54649)
0812 04:01:22.015384 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:22.015435 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":74,"threads_started":1}
I20260812 04:01:22.826268 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:22.825991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:47697)
0812 04:01:22.826158 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:22.826207 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260812 04:01:22.882426 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:22.882133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:55411)
0812 04:01:22.882325 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:22.882375 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
I20260812 04:01:23.016383 18853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:23.016106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:59159)
0812 04:01:23.016282 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:23.016331 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260812 04:01:23.742368 18854 raft_consensus.cc:482] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:01:23.742471 18854 raft_consensus.cc:504] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce616a372a54fe98ed432a263a734b4" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44585 } } peers { permanent_uuid: "2a49b75f562b47e78bce2dbadc470d09" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 41829 } } peers { permanent_uuid: "7c2e85f133ae41ceb4ec6549f9e8701e" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45939 } }
I20260812 04:01:23.742656 18854 leader_election.cc:288] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585), 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829)
W20260812 04:01:23.743141 18389 leader_election.cc:341] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9ce616a372a54fe98ed432a263a734b4 (127.31.43.195:44585): Not found: Tablet not found: 0c72a47f069d49749fde740d163afb6f
W20260812 04:01:23.743206 18386 leader_election.cc:341] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a49b75f562b47e78bce2dbadc470d09 (127.31.43.194:41829): Not found: Tablet not found: 0c72a47f069d49749fde740d163afb6f
I20260812 04:01:23.743234 18389 leader_election.cc:302] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c2e85f133ae41ceb4ec6549f9e8701e; no voters: 2a49b75f562b47e78bce2dbadc470d09, 9ce616a372a54fe98ed432a263a734b4
I20260812 04:01:23.743343 18854 raft_consensus.cc:2726] T 0c72a47f069d49749fde740d163afb6f P 7c2e85f133ae41ceb4ec6549f9e8701e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 04:01:23.827126 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:23.826836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:55425)
0812 04:01:23.827021 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:23.827069 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
I20260812 04:01:23.883267 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:23.882992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:52275)
0812 04:01:23.883172 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:23.883216 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":89,"threads_started":1}
I20260812 04:01:24.017221 18857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:24.016944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:35323)
0812 04:01:24.017128 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:24.017172 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":92,"threads_started":1}
I20260812 04:01:24.828068 18858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:24.827776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:43739)
0812 04:01:24.827968 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:24.828017 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260812 04:01:24.884083 18859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:24.883815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:37563)
0812 04:01:24.883972 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:24.884010 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260812 04:01:25.018152 18860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:25.017839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:54659)
0812 04:01:25.018029 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:25.018073 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":75,"threads_started":1}
I20260812 04:01:25.828948 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:25.828666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:55033)
0812 04:01:25.828850 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:25.828895 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":72,"threads_started":1}
I20260812 04:01:25.884833 18863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:25.884591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:40909)
0812 04:01:25.884745 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:25.884785 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260812 04:01:26.018922 18865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:26.018634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:37581)
0812 04:01:26.018837 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:26.018874 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":133,"threads_started":1}
I20260812 04:01:26.829923 18867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:26.829657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:37523)
0812 04:01:26.829823 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:26.829873 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":69,"threads_started":1}
I20260812 04:01:26.885589 18868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:26.885342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:58757)
0812 04:01:26.885500 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:26.885539 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260812 04:01:27.019749 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:27.019508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:53697)
0812 04:01:27.019650 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:27.019698 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260812 04:01:27.830843 18870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:27.830588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.193:37211)
0812 04:01:27.830758 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:27.830795 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":67,"threads_started":1}
I20260812 04:01:27.886476 18871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:27.886243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.194:57003)
0812 04:01:27.886395 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:27.886428 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260812 04:01:28.018112 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 18330
I20260812 04:01:28.022423 18872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:28.021776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33795 (local address 127.31.43.195:35357)
0812 04:01:28.022323 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:28.022360 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.254:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":297,"threads_started":1}
I20260812 04:01:28.027503 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 18519
I20260812 04:01:28.035625 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 18667
2026-08-12T04:01:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58577 ms)
[----------] 30 tests from SecurityITest (246179 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:28 dist-test-slave-8tdl krb5kdc[18876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:28 dist-test-slave-8tdl krb5kdc[18876](info): set up 2 sockets
Aug 12 04:01:28 dist-test-slave-8tdl krb5kdc[18876](info): commencing operation
krb5kdc: starting...
W20260812 04:01:29.183506 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:29 dist-test-slave-8tdl krb5kdc[18876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:01:29.318856 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:42465
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:42465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:29.406208 18891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:29.406514 18891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:29.406553 18891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:29.408200 18891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:29.408273 18891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:29.408298 18891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:29.408330 18891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:29.408347 18891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:29.410106 18891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:42465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:42465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:29.410398 18891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:29.410660 18891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:29.416494 18896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:29.416615 18897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:29.416626 18899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:29.416517 18891 server_base.cc:1034] running on GCE node
I20260812 04:01:29.417124 18891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:29.417402 18891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:29.418572 18891 hybrid_clock.cc:648] HybridClock initialized: now 1786507289418555 us; error 34 us; skew 500 ppm
Aug 12 04:01:29 dist-test-slave-8tdl krb5kdc[18876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507289, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:29.421512 18891 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:01:29.422076 18891 webserver.cc:460] Webserver started at http://127.31.43.254:43745/ using document root <none> and password file <none>
I20260812 04:01:29.422324 18891 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:29.422372 18891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:29.422526 18891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:29.423722 18891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "f145925e03ac4324ad3a08dbb06a8787"
format_stamp: "Formatted at 2026-08-12 04:01:29 on dist-test-slave-8tdl"
server_key: "8268057f3b23c108bdd1aba4287ac40d"
server_key_iv: "d12e5b35da454b6c363851c67c38acd8"
server_key_version: "encryptionkey@0"
I20260812 04:01:29.424166 18891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "f145925e03ac4324ad3a08dbb06a8787"
format_stamp: "Formatted at 2026-08-12 04:01:29 on dist-test-slave-8tdl"
server_key: "8268057f3b23c108bdd1aba4287ac40d"
server_key_iv: "d12e5b35da454b6c363851c67c38acd8"
server_key_version: "encryptionkey@0"
I20260812 04:01:29.425717 18891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:29.426527 18906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:29.426761 18891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:29.426872 18891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "f145925e03ac4324ad3a08dbb06a8787"
format_stamp: "Formatted at 2026-08-12 04:01:29 on dist-test-slave-8tdl"
server_key: "8268057f3b23c108bdd1aba4287ac40d"
server_key_iv: "d12e5b35da454b6c363851c67c38acd8"
server_key_version: "encryptionkey@0"
I20260812 04:01:29.426957 18891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:29.440177 18891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:29.442396 18891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:29.442584 18891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:29.447216 18891 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:42465
I20260812 04:01:29.447297 18965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:42465 every 8 connection(s)
I20260812 04:01:29.447700 18891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:29.448653 18966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:29.451174 18966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787: Bootstrap starting.
I20260812 04:01:29.451768 18966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:29.452037 18966 log.cc:824] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:29.452798 18966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787: No bootstrap required, opened a new log
I20260812 04:01:29.454429 18966 raft_consensus.cc:348] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f145925e03ac4324ad3a08dbb06a8787" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42465 } }
I20260812 04:01:29.454584 18966 raft_consensus.cc:374] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:29.454608 18966 raft_consensus.cc:729] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f145925e03ac4324ad3a08dbb06a8787, State: Initialized, Role: FOLLOWER
I20260812 04:01:29.454790 18966 consensus_queue.cc:260] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f145925e03ac4324ad3a08dbb06a8787" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42465 } }
I20260812 04:01:29.454893 18966 raft_consensus.cc:388] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:29.454919 18966 raft_consensus.cc:482] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:29.454952 18966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:29.455489 18966 raft_consensus.cc:504] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f145925e03ac4324ad3a08dbb06a8787" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42465 } }
I20260812 04:01:29.455619 18966 leader_election.cc:302] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f145925e03ac4324ad3a08dbb06a8787; no voters:
I20260812 04:01:29.455716 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 18891
I20260812 04:01:29.455854 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:29.455879 18966 leader_election.cc:288] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:29.455931 18971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:29.456095 31919 external_mini_cluster.cc:1428] Setting key a8422f551109eb2297fb818e0250ee27
I20260812 04:01:29.456161 18971 raft_consensus.cc:686] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [term 1 LEADER]: Becoming Leader. State: Replica: f145925e03ac4324ad3a08dbb06a8787, State: Running, Role: LEADER
I20260812 04:01:29.456173 18966 sys_catalog.cc:564] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:29.456298 18971 consensus_queue.cc:237] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f145925e03ac4324ad3a08dbb06a8787" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42465 } }
I20260812 04:01:29.456807 18973 sys_catalog.cc:455] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f145925e03ac4324ad3a08dbb06a8787. Latest consensus state: current_term: 1 leader_uuid: "f145925e03ac4324ad3a08dbb06a8787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f145925e03ac4324ad3a08dbb06a8787" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42465 } } }
I20260812 04:01:29.456912 18973 sys_catalog.cc:458] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:29.456959 18972 sys_catalog.cc:455] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f145925e03ac4324ad3a08dbb06a8787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f145925e03ac4324ad3a08dbb06a8787" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42465 } } }
I20260812 04:01:29.457052 18972 sys_catalog.cc:458] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:29.457172 18981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:29.457764 18981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:29.460083 18981 catalog_manager.cc:1347] Generated new cluster ID: a288a56b40894c7d868b7fcd491cd269
I20260812 04:01:29.460139 18981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 04:01:29 dist-test-slave-8tdl krb5kdc[18876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:29.468920 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:29.457184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33100 (local address 127.31.43.254:42465)
0812 04:01:29.457389 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:29.457399 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:29.457717 (+ 318us) server_negotiation.cc:407] Connection header received
0812 04:01:29.457848 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:29.457856 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:29.457922 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:29.458092 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:29.459033 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:29.460310 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:29.461402 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:29.461999 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:29.462105 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:29.464989 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:29.464995 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:29.464995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:29.465008 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:29.467157 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:29.467560 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:29.467564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:29.467564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:29.467715 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:29.467927 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:29.467928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:29.467928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:29.468291 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:29.468463 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:29.468534 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 04:01:29.468685 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260812 04:01:29.489511 18981 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:29.490140 18981 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:29.495340 18981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787: Generated new TSK 0
I20260812 04:01:29.495553 18981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 04:01:29.531980 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42465
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:29.620442 18997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:29.620693 18997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:29.620735 18997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:29.621932 18997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:29.622004 18997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:29.622066 18997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:01:29.623747 18997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:29.624058 18997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:29.624380 18997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:29.630108 19005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:29.630178 19002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:29.630182 18997 server_base.cc:1034] running on GCE node
W20260812 04:01:29.630183 19003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:29.630563 18997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:29.630897 18997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:29.632069 18997 hybrid_clock.cc:648] HybridClock initialized: now 1786507289632065 us; error 40 us; skew 500 ppm
Aug 12 04:01:29 dist-test-slave-8tdl krb5kdc[18876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507289, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:29.634819 18997 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 04:01:29.635336 18997 webserver.cc:460] Webserver started at http://127.31.43.193:37643/ using document root <none> and password file <none>
I20260812 04:01:29.635600 18997 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:29.635682 18997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:29.635834 18997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:29.636942 18997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "a272b800846543a68d783c83fa3aad84"
format_stamp: "Formatted at 2026-08-12 04:01:29 on dist-test-slave-8tdl"
server_key: "baafde732764bcb8ff49a3a54108cdf7"
server_key_iv: "bef87b6c6a000ba4d6f813ead2f38240"
server_key_version: "encryptionkey@0"
I20260812 04:01:29.637324 18997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "a272b800846543a68d783c83fa3aad84"
format_stamp: "Formatted at 2026-08-12 04:01:29 on dist-test-slave-8tdl"
server_key: "baafde732764bcb8ff49a3a54108cdf7"
server_key_iv: "bef87b6c6a000ba4d6f813ead2f38240"
server_key_version: "encryptionkey@0"
I20260812 04:01:29.638818 18997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:29.639654 19012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:29.639863 18997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:29.639976 18997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "a272b800846543a68d783c83fa3aad84"
format_stamp: "Formatted at 2026-08-12 04:01:29 on dist-test-slave-8tdl"
server_key: "baafde732764bcb8ff49a3a54108cdf7"
server_key_iv: "bef87b6c6a000ba4d6f813ead2f38240"
server_key_version: "encryptionkey@0"
I20260812 04:01:29.640054 18997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:29.666312 18997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:29.669036 18997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:29.669231 18997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:29.669502 18997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:29.669888 18997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:29.669948 18997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:29.669986 18997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:29.670030 18997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:29.676455 18997 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:40061
I20260812 04:01:29.676541 19142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:40061 every 8 connection(s)
I20260812 04:01:29.676919 18997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 04:01:29.678911 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 18997
I20260812 04:01:29.679031 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 04:01:29.679302 31919 external_mini_cluster.cc:1428] Setting key 9085f4590d4e9692d563898f6b22e7dd
Aug 12 04:01:29 dist-test-slave-8tdl krb5kdc[18876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507289, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:29.688374 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:29.677853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:37173 (local address 127.31.43.254:42465)
0812 04:01:29.677961 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:29.677963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:29.678140 (+ 177us) server_negotiation.cc:407] Connection header received
0812 04:01:29.678763 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:29.678764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:29.678781 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:29.678842 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:29.680121 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:29.680859 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:29.681955 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:29.682181 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:29.682278 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:29.684905 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:29.684907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:29.684908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:29.684918 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:29.686547 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:29.687279 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:29.687280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:29.687281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:29.687340 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:29.687658 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:29.687658 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:29.687658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:29.687772 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:29.687822 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:29.688219 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 04:01:29.688304 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 04:01:29.688658 19145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:29.677757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42465 (local address 127.31.43.193:37173)
0812 04:01:29.677995 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:29.678043 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:29.678605 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:29.678923 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:29.678927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:29.679749 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:29.680011 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:29.680019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:29.680969 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:29.680970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:29.681857 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:29.681860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:29.682293 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:29.682294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:29.682463 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:29.683100 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:01:29.683113 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:29.684795 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:29.686701 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:29.686712 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:29.686715 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:29.687189 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:29.687417 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:29.687418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:29.687418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:29.687581 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:29.687842 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:29.687849 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:29.688052 (+ 203us) client_negotiation.cc:769] Sending connection context
0812 04:01:29.688178 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 04:01:29.688390 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":115,"threads_started":1}
I20260812 04:01:29.689226 19143 heartbeater.cc:347] Connected to a master server at 127.31.43.254:42465
I20260812 04:01:29.689324 19143 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:29.689538 19143 heartbeater.cc:511] Master 127.31.43.254:42465 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
I20260812 04:01:29.690167 18923 ts_manager.cc:194] Registered new tserver with Master: a272b800846543a68d783c83fa3aad84 (127.31.43.193:40061)
I20260812 04:01:29.690935 18923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:37173
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 04:01:29.711530 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42465
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:29.801638 19148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:29.801868 19148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:29.801896 19148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:29.803097 19148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:29.803140 19148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:29.803167 19148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:01:29.804695 19148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:29.804893 19148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:29.805095 19148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:29.810331 19155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:29.810331 19157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:29.810379 19154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:29.810659 19148 server_base.cc:1034] running on GCE node
I20260812 04:01:29.810914 19148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:29.811195 19148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:29.812366 19148 hybrid_clock.cc:648] HybridClock initialized: now 1786507289812359 us; error 38 us; skew 500 ppm
Aug 12 04:01:29 dist-test-slave-8tdl krb5kdc[18876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507289, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:29.814929 19148 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 04:01:29.815412 19148 webserver.cc:460] Webserver started at http://127.31.43.194:36489/ using document root <none> and password file <none>
I20260812 04:01:29.815673 19148 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:29.815747 19148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:29.815898 19148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:29.817004 19148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "f24d11805557490f80148e24d431e5e4"
format_stamp: "Formatted at 2026-08-12 04:01:29 on dist-test-slave-8tdl"
server_key: "f43ce207ec7164f4c29940637392baf7"
server_key_iv: "ea474b0781e881d0af721e849ed2dee7"
server_key_version: "encryptionkey@0"
I20260812 04:01:29.817400 19148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "f24d11805557490f80148e24d431e5e4"
format_stamp: "Formatted at 2026-08-12 04:01:29 on dist-test-slave-8tdl"
server_key: "f43ce207ec7164f4c29940637392baf7"
server_key_iv: "ea474b0781e881d0af721e849ed2dee7"
server_key_version: "encryptionkey@0"
I20260812 04:01:29.818928 19148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 04:01:29.819813 19164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:29.820039 19148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:29.820189 19148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "f24d11805557490f80148e24d431e5e4"
format_stamp: "Formatted at 2026-08-12 04:01:29 on dist-test-slave-8tdl"
server_key: "f43ce207ec7164f4c29940637392baf7"
server_key_iv: "ea474b0781e881d0af721e849ed2dee7"
server_key_version: "encryptionkey@0"
I20260812 04:01:29.820269 19148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:29.835083 19148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:29.837651 19148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:29.837815 19148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:29.838094 19148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:29.838435 19148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:29.838495 19148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:29.838538 19148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:29.838578 19148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:29.844863 19148 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:35147
I20260812 04:01:29.844998 19294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:35147 every 8 connection(s)
I20260812 04:01:29.845300 19148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:01:29.848168 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 19148
I20260812 04:01:29.848282 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 04:01:29.848500 31919 external_mini_cluster.cc:1428] Setting key de16c82dc65b4edee8b36a4959b890dd
Aug 12 04:01:29 dist-test-slave-8tdl krb5kdc[18876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507289, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:29.856388 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:29.846160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:35603 (local address 127.31.43.254:42465)
0812 04:01:29.846212 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:29.846214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:29.846385 (+ 171us) server_negotiation.cc:407] Connection header received
0812 04:01:29.847032 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:29.847033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:29.847043 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:29.847127 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:29.848133 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:29.848842 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:29.849969 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:29.850297 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:29.850379 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:29.853000 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:29.853001 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:29.853001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:29.853010 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:29.854778 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:29.855203 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:29.855205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:29.855206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:29.855277 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:29.855523 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:29.855524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:29.855525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:29.855699 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:29.855780 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:29.856146 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 04:01:29.856246 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 04:01:29.856494 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:29.846060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42465 (local address 127.31.43.194:35603)
0812 04:01:29.846263 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:29.846308 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:29.846939 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:29.847206 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:29.847209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:29.847790 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:29.847990 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:29.847994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:29.848926 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:29.848927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:29.849876 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:29.849879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:29.850396 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:29.850397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:29.850516 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:29.851212 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:01:29.851225 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:29.852900 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:29.854862 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:29.854878 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:29.854880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:29.855121 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:29.855353 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:29.855353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:29.855353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:29.855456 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:29.855774 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:29.855778 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:29.855897 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 04:01:29.856115 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 04:01:29.856260 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":126,"threads_started":1}
I20260812 04:01:29.857071 19295 heartbeater.cc:347] Connected to a master server at 127.31.43.254:42465
I20260812 04:01:29.857182 19295 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:29.857374 19295 heartbeater.cc:511] Master 127.31.43.254:42465 requested a full tablet report, sending...
I20260812 04:01:29.857877 18923 ts_manager.cc:194] Registered new tserver with Master: f24d11805557490f80148e24d431e5e4 (127.31.43.194:35147)
I20260812 04:01:29.858451 18923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:35603
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 04:01:29.881244 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42465
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:29.969798 19300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:29.970027 19300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:29.970052 19300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:29.971257 19300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:29.971302 19300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:29.971328 19300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:01:29.972777 19300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:29.972980 19300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:29.973176 19300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:29.978889 19306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:29.978948 19300 server_base.cc:1034] running on GCE node
W20260812 04:01:29.978896 19309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:29.978901 19307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:29.979329 19300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:29.979606 19300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:29.980767 19300 hybrid_clock.cc:648] HybridClock initialized: now 1786507289980744 us; error 34 us; skew 500 ppm
Aug 12 04:01:29 dist-test-slave-8tdl krb5kdc[18876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507289, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:29.983556 19300 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 04:01:29.984036 19300 webserver.cc:460] Webserver started at http://127.31.43.195:39691/ using document root <none> and password file <none>
I20260812 04:01:29.984272 19300 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:29.984339 19300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:29.984506 19300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:29.985599 19300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "b19e86096cc14e35b19f56fb8841d0bc"
format_stamp: "Formatted at 2026-08-12 04:01:29 on dist-test-slave-8tdl"
server_key: "d8b9a997641b4a78545fed5132179ccb"
server_key_iv: "3e188bfa26080f640edc779e4eba37c4"
server_key_version: "encryptionkey@0"
I20260812 04:01:29.985973 19300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "b19e86096cc14e35b19f56fb8841d0bc"
format_stamp: "Formatted at 2026-08-12 04:01:29 on dist-test-slave-8tdl"
server_key: "d8b9a997641b4a78545fed5132179ccb"
server_key_iv: "3e188bfa26080f640edc779e4eba37c4"
server_key_version: "encryptionkey@0"
I20260812 04:01:29.987464 19300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 04:01:29.988267 19316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:29.988564 19300 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:29.988675 19300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "b19e86096cc14e35b19f56fb8841d0bc"
format_stamp: "Formatted at 2026-08-12 04:01:29 on dist-test-slave-8tdl"
server_key: "d8b9a997641b4a78545fed5132179ccb"
server_key_iv: "3e188bfa26080f640edc779e4eba37c4"
server_key_version: "encryptionkey@0"
I20260812 04:01:29.988751 19300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:30.008140 19300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:30.010617 19300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:30.010845 19300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:30.011060 19300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:30.011337 19300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:30.011368 19300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:30.011391 19300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:30.011404 19300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:30.017701 19300 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:43095
I20260812 04:01:30.017784 19443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:43095 every 8 connection(s)
I20260812 04:01:30.018114 19300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
Aug 12 04:01:30 dist-test-slave-8tdl krb5kdc[18876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507289, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:30.027930 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 19300
I20260812 04:01:30.028024 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 04:01:30.028225 31919 external_mini_cluster.cc:1428] Setting key f29383bd4e3160527e75c77b183db6e1
I20260812 04:01:30.029376 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.019029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:48967 (local address 127.31.43.254:42465)
0812 04:01:30.019109 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:30.019112 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:30.019304 (+ 192us) server_negotiation.cc:407] Connection header received
0812 04:01:30.019826 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:30.019827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:30.019836 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:30.019902 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:30.021058 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.021663 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.022923 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.023188 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.023251 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.025779 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:30.025781 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:30.025781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:30.025789 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:30.027359 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:30.027890 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:30.027891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:30.027891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:30.028042 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:30.028393 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:30.028395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:30.028395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:30.028564 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:30.028634 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:30.029164 (+ 530us) server_negotiation.cc:299] Negotiation successful
0812 04:01:30.029269 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 04:01:30.029618 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.018949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42465 (local address 127.31.43.195:48967)
0812 04:01:30.019154 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:30.019201 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:30.019737 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:30.020027 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:30.020031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:30.020725 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:30.020953 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.020958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.021761 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.021762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.022832 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.022835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.023294 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.023295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.023422 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.024019 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:01:30.024028 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:30.025673 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:30.027486 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:30.027495 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:30.027498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:30.027793 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:30.028128 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:30.028129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:30.028130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:30.028290 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:30.028623 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:30.028631 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:30.028800 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 04:01:30.029185 (+ 385us) client_negotiation.cc:240] Negotiation successful
0812 04:01:30.029370 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260812 04:01:30.030102 19444 heartbeater.cc:347] Connected to a master server at 127.31.43.254:42465
I20260812 04:01:30.030202 19444 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:30.030360 19444 heartbeater.cc:511] Master 127.31.43.254:42465 requested a full tablet report, sending...
I20260812 04:01:30.030791 18923 ts_manager.cc:194] Registered new tserver with Master: b19e86096cc14e35b19f56fb8841d0bc (127.31.43.195:43095)
I20260812 04:01:30.031195 18923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:48967
I20260812 04:01:30.040181 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 04:01:30.048321 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.041687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33106 (local address 127.31.43.254:42465)
0812 04:01:30.041780 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:30.041782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:30.041872 (+ 90us) server_negotiation.cc:407] Connection header received
0812 04:01:30.041961 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:30.041962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:30.041973 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:30.042031 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:30.042796 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.043322 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.044136 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.044351 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.044444 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.045673 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:30.045675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:30.045676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:30.045683 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:30.047083 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:30.047430 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:30.047432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:30.047432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:30.047496 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:30.047722 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:30.047723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:30.047723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:30.047892 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:30.048009 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:30.048120 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 04:01:30.048209 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 04:01:30.050120 18923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 04:01:30.052003 18923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 04:01:30.059264 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.054577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:35147 (local address 127.0.0.1:33974)
0812 04:01:30.054908 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:30.054920 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:30.054996 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:30.055721 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:30.055722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:30.055724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:30.055762 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.055764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.056730 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.056731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.057936 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.057939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.058924 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.058925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.059012 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.059028 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 04:01:30.059100 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 04:01:30.059163 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":130,"threads_started":1}
I20260812 04:01:30.059389 19460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.054779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59172 (local address 127.31.43.195:43095)
0812 04:01:30.055253 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:30.055258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:30.055267 (+ 9us) server_negotiation.cc:407] Connection header received
0812 04:01:30.055310 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:30.055312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:30.055327 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:30.055403 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:30.055607 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.056805 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.058333 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.059002 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.059098 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.059154 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:30.059251 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 04:01:30.059295 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":117,"threads_started":1}
I20260812 04:01:30.059667 19461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.054907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36154 (local address 127.31.43.193:40061)
0812 04:01:30.055129 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:30.055134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:30.055235 (+ 101us) server_negotiation.cc:407] Connection header received
0812 04:01:30.055284 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:30.055287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:30.055305 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:30.055377 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:30.055649 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.056804 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.058300 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.059063 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.059147 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.059205 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:30.059516 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 04:01:30.059566 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":97,"threads_started":1}
I20260812 04:01:30.059717 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.055029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33974 (local address 127.31.43.194:35147)
0812 04:01:30.055545 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:30.055549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:30.055559 (+ 10us) server_negotiation.cc:407] Connection header received
0812 04:01:30.055600 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:30.055602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:30.055616 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:30.055722 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:30.055821 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.056631 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.058031 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.058830 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.058909 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.058970 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:30.059617 (+ 647us) server_negotiation.cc:299] Negotiation successful
0812 04:01:30.059654 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":92,"threads_started":1}
I20260812 04:01:30.059880 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.054616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:40061 (local address 127.0.0.1:36154)
0812 04:01:30.055122 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:30.055132 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:30.055184 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:30.055474 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:30.055476 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:30.055478 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:30.055538 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.055539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.056928 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.056929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.058202 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.058206 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.059239 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.059240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.059291 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.059294 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:01:30.059342 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 04:01:30.059801 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":6,"thread_start_us":104,"threads_started":1}
I20260812 04:01:30.060278 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.054573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:43095 (local address 127.0.0.1:59172)
0812 04:01:30.054786 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:30.054832 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:30.055002 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:30.055421 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:30.055424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:30.055427 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:30.055516 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.055518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.056928 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.056929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.058244 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.058245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.059102 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.059103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.059168 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.059172 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:01:30.059236 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 04:01:30.059279 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":200,"thread_start_us":71,"threads_started":1}
I20260812 04:01:30.060531 19217 tablet_service.cc:1467] Processing CreateTablet for tablet a76d543997024a949d505cf045afc6b5 (DEFAULT_TABLE table=test-table [id=9c06d1a8f7a44778b05245a54605260d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:30.060590 19066 tablet_service.cc:1467] Processing CreateTablet for tablet a76d543997024a949d505cf045afc6b5 (DEFAULT_TABLE table=test-table [id=9c06d1a8f7a44778b05245a54605260d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:30.060968 19066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a76d543997024a949d505cf045afc6b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:30.060968 19217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a76d543997024a949d505cf045afc6b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:30.063366 19371 tablet_service.cc:1467] Processing CreateTablet for tablet a76d543997024a949d505cf045afc6b5 (DEFAULT_TABLE table=test-table [id=9c06d1a8f7a44778b05245a54605260d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:30.063736 19371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a76d543997024a949d505cf045afc6b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:30.064824 19465 tablet_bootstrap.cc:492] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84: Bootstrap starting.
I20260812 04:01:30.065606 19465 tablet_bootstrap.cc:654] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:30.066303 19465 log.cc:824] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:30.066658 19463 tablet_bootstrap.cc:492] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4: Bootstrap starting.
I20260812 04:01:30.067384 19465 tablet_bootstrap.cc:492] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84: No bootstrap required, opened a new log
I20260812 04:01:30.067399 19467 tablet_bootstrap.cc:492] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc: Bootstrap starting.
I20260812 04:01:30.067418 19463 tablet_bootstrap.cc:654] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:30.067574 19465 ts_tablet_manager.cc:1397] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 04:01:30.067927 19463 log.cc:824] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:30.068079 19467 tablet_bootstrap.cc:654] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:30.068634 19467 log.cc:824] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:30.068662 19463 tablet_bootstrap.cc:492] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4: No bootstrap required, opened a new log
I20260812 04:01:30.068747 19463 ts_tablet_manager.cc:1397] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 04:01:30.069440 19467 tablet_bootstrap.cc:492] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc: No bootstrap required, opened a new log
I20260812 04:01:30.069518 19467 ts_tablet_manager.cc:1397] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 04:01:30.069916 19465 raft_consensus.cc:348] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19e86096cc14e35b19f56fb8841d0bc" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 43095 } } peers { permanent_uuid: "f24d11805557490f80148e24d431e5e4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35147 } } peers { permanent_uuid: "a272b800846543a68d783c83fa3aad84" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40061 } }
I20260812 04:01:30.070123 19465 raft_consensus.cc:374] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:30.070160 19465 raft_consensus.cc:729] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a272b800846543a68d783c83fa3aad84, State: Initialized, Role: FOLLOWER
I20260812 04:01:30.070291 19465 consensus_queue.cc:260] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19e86096cc14e35b19f56fb8841d0bc" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 43095 } } peers { permanent_uuid: "f24d11805557490f80148e24d431e5e4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35147 } } peers { permanent_uuid: "a272b800846543a68d783c83fa3aad84" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40061 } }
I20260812 04:01:30.070539 19463 raft_consensus.cc:348] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19e86096cc14e35b19f56fb8841d0bc" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 43095 } } peers { permanent_uuid: "f24d11805557490f80148e24d431e5e4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35147 } } peers { permanent_uuid: "a272b800846543a68d783c83fa3aad84" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40061 } }
I20260812 04:01:30.070716 19463 raft_consensus.cc:374] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:30.070772 19463 raft_consensus.cc:729] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f24d11805557490f80148e24d431e5e4, State: Initialized, Role: FOLLOWER
I20260812 04:01:30.070899 19463 consensus_queue.cc:260] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19e86096cc14e35b19f56fb8841d0bc" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 43095 } } peers { permanent_uuid: "f24d11805557490f80148e24d431e5e4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35147 } } peers { permanent_uuid: "a272b800846543a68d783c83fa3aad84" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40061 } }
I20260812 04:01:30.071004 19465 ts_tablet_manager.cc:1428] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 04:01:30.071153 19143 heartbeater.cc:503] Master 127.31.43.254:42465 was elected leader, sending a full tablet report...
I20260812 04:01:30.071197 19463 ts_tablet_manager.cc:1428] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 04:01:30.071434 19295 heartbeater.cc:503] Master 127.31.43.254:42465 was elected leader, sending a full tablet report...
I20260812 04:01:30.071640 19467 raft_consensus.cc:348] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19e86096cc14e35b19f56fb8841d0bc" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 43095 } } peers { permanent_uuid: "f24d11805557490f80148e24d431e5e4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35147 } } peers { permanent_uuid: "a272b800846543a68d783c83fa3aad84" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40061 } }
I20260812 04:01:30.071821 19467 raft_consensus.cc:374] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:30.071861 19467 raft_consensus.cc:729] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b19e86096cc14e35b19f56fb8841d0bc, State: Initialized, Role: FOLLOWER
I20260812 04:01:30.071970 19467 consensus_queue.cc:260] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19e86096cc14e35b19f56fb8841d0bc" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 43095 } } peers { permanent_uuid: "f24d11805557490f80148e24d431e5e4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35147 } } peers { permanent_uuid: "a272b800846543a68d783c83fa3aad84" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40061 } }
I20260812 04:01:30.072275 19467 ts_tablet_manager.cc:1428] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 04:01:30.072329 19444 heartbeater.cc:503] Master 127.31.43.254:42465 was elected leader, sending a full tablet report...
W20260812 04:01:30.095927 19296 tablet.cc:2369] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 04:01:30.177763 19144 tablet.cc:2369] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 04:01:30.268790 19445 tablet.cc:2369] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:01:30.355186 19473 raft_consensus.cc:482] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:01:30.355358 19473 raft_consensus.cc:504] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19e86096cc14e35b19f56fb8841d0bc" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 43095 } } peers { permanent_uuid: "f24d11805557490f80148e24d431e5e4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35147 } } peers { permanent_uuid: "a272b800846543a68d783c83fa3aad84" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40061 } }
I20260812 04:01:30.355847 19473 leader_election.cc:288] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f24d11805557490f80148e24d431e5e4 (127.31.43.194:35147), a272b800846543a68d783c83fa3aad84 (127.31.43.193:40061)
I20260812 04:01:30.359150 19461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.356174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:39923 (local address 127.31.43.193:40061)
0812 04:01:30.356223 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:30.356225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:30.356237 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:01:30.356294 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:30.356299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:30.356328 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:30.356386 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:30.356580 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.357202 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.358294 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.358755 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.358850 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.358861 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:30.359032 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 04:01:30.359076 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 04:01:30.359150 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.356079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:40061 (local address 127.31.43.195:39923)
0812 04:01:30.356143 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:30.356157 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:30.356211 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:30.356391 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:30.356391 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:30.356393 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:30.356481 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.356483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.357303 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.357304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.358200 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.358203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.358850 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.358851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.358940 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.358944 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:01:30.359012 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 04:01:30.359069 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 04:01:30.359439 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.356089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:35147 (local address 127.31.43.195:38329)
0812 04:01:30.356305 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:30.356317 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:30.356412 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:30.356624 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:30.356625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:30.356626 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:30.356698 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.356700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.357570 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.357571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.358439 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.358441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.359122 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.359123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.359203 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.359207 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:01:30.359302 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 04:01:30.359356 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":3,"thread_start_us":103,"threads_started":1}
I20260812 04:01:30.359469 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.356132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:38329 (local address 127.31.43.194:35147)
0812 04:01:30.356204 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:30.356206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:30.356397 (+ 191us) server_negotiation.cc:407] Connection header received
0812 04:01:30.356472 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:30.356474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:30.356495 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:30.356583 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:30.356770 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.357476 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.358531 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.359023 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.359121 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.359129 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:30.359339 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 04:01:30.359398 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 04:01:30.359630 19090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76d543997024a949d505cf045afc6b5" candidate_uuid: "b19e86096cc14e35b19f56fb8841d0bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a272b800846543a68d783c83fa3aad84" is_pre_election: true
I20260812 04:01:30.359766 19239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76d543997024a949d505cf045afc6b5" candidate_uuid: "b19e86096cc14e35b19f56fb8841d0bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24d11805557490f80148e24d431e5e4" is_pre_election: true
I20260812 04:01:30.359830 19090 raft_consensus.cc:2454] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b19e86096cc14e35b19f56fb8841d0bc in term 0.
I20260812 04:01:30.359899 19239 raft_consensus.cc:2454] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b19e86096cc14e35b19f56fb8841d0bc in term 0.
I20260812 04:01:30.360113 19317 leader_election.cc:302] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b19e86096cc14e35b19f56fb8841d0bc, f24d11805557490f80148e24d431e5e4; no voters:
I20260812 04:01:30.360309 19473 raft_consensus.cc:2781] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 04:01:30.360404 19473 raft_consensus.cc:482] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 04:01:30.360438 19473 raft_consensus.cc:3037] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:30.361233 19473 raft_consensus.cc:504] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19e86096cc14e35b19f56fb8841d0bc" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 43095 } } peers { permanent_uuid: "f24d11805557490f80148e24d431e5e4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35147 } } peers { permanent_uuid: "a272b800846543a68d783c83fa3aad84" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40061 } }
I20260812 04:01:30.361413 19473 leader_election.cc:288] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [CANDIDATE]: Term 1 election: Requested vote from peers f24d11805557490f80148e24d431e5e4 (127.31.43.194:35147), a272b800846543a68d783c83fa3aad84 (127.31.43.193:40061)
I20260812 04:01:30.361599 19239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76d543997024a949d505cf045afc6b5" candidate_uuid: "b19e86096cc14e35b19f56fb8841d0bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24d11805557490f80148e24d431e5e4"
I20260812 04:01:30.361613 19090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76d543997024a949d505cf045afc6b5" candidate_uuid: "b19e86096cc14e35b19f56fb8841d0bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a272b800846543a68d783c83fa3aad84"
I20260812 04:01:30.361699 19239 raft_consensus.cc:3037] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:30.361699 19090 raft_consensus.cc:3037] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:30.362664 19090 raft_consensus.cc:2454] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b19e86096cc14e35b19f56fb8841d0bc in term 1.
I20260812 04:01:30.362664 19239 raft_consensus.cc:2454] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b19e86096cc14e35b19f56fb8841d0bc in term 1.
I20260812 04:01:30.362871 19317 leader_election.cc:302] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b19e86096cc14e35b19f56fb8841d0bc, f24d11805557490f80148e24d431e5e4; no voters:
I20260812 04:01:30.362977 19473 raft_consensus.cc:2781] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:30.363135 19473 raft_consensus.cc:686] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 1 LEADER]: Becoming Leader. State: Replica: b19e86096cc14e35b19f56fb8841d0bc, State: Running, Role: LEADER
I20260812 04:01:30.363252 19473 consensus_queue.cc:237] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19e86096cc14e35b19f56fb8841d0bc" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 43095 } } peers { permanent_uuid: "f24d11805557490f80148e24d431e5e4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35147 } } peers { permanent_uuid: "a272b800846543a68d783c83fa3aad84" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40061 } }
I20260812 04:01:30.364190 18922 catalog_manager.cc:5495] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc reported cstate change: term changed from 0 to 1, leader changed from <none> to b19e86096cc14e35b19f56fb8841d0bc (127.31.43.195). New cstate: current_term: 1 leader_uuid: "b19e86096cc14e35b19f56fb8841d0bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b19e86096cc14e35b19f56fb8841d0bc" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 43095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f24d11805557490f80148e24d431e5e4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a272b800846543a68d783c83fa3aad84" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40061 } health_report { overall_health: UNKNOWN } } }
I20260812 04:01:30.413818 19460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.410334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59184 (local address 127.31.43.195:43095)
0812 04:01:30.410382 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:30.410384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:30.410431 (+ 47us) server_negotiation.cc:407] Connection header received
0812 04:01:30.410607 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:30.410609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:30.410653 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:30.410730 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:01:30.411026 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.411774 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.413044 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.413277 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.413341 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.413488 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:01:30.413603 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:01:30.413664 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:30.413710 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 04:01:30.413730 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 04:01:30.416038 19090 raft_consensus.cc:1261] T a76d543997024a949d505cf045afc6b5 P a272b800846543a68d783c83fa3aad84 [term 1 FOLLOWER]: Refusing update from remote peer b19e86096cc14e35b19f56fb8841d0bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:01:30.416095 19239 raft_consensus.cc:1261] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4 [term 1 FOLLOWER]: Refusing update from remote peer b19e86096cc14e35b19f56fb8841d0bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:01:30.416493 19475 consensus_queue.cc:1035] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a272b800846543a68d783c83fa3aad84" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:01:30.416625 19473 consensus_queue.cc:1035] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f24d11805557490f80148e24d431e5e4" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 35147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:01:30.419898 19478 mvcc.cc:205] Tried to move back new op lower bound from 7317533861540728832 to 7317533861328642048. Current Snapshot: MvccSnapshot[applied={T|T < 7317533861540728832}]
I20260812 04:01:30.420933 19479 mvcc.cc:205] Tried to move back new op lower bound from 7317533861540728832 to 7317533861328642048. Current Snapshot: MvccSnapshot[applied={T|T < 7317533861540728832}]
I20260812 04:01:30.422726 18923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:01:30.422839 18923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:01:30.423758 18923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f145925e03ac4324ad3a08dbb06a8787: Sending DeleteTablet for 3 replicas of tablet a76d543997024a949d505cf045afc6b5
I20260812 04:01:30.424131 19371 tablet_service.cc:1514] Processing DeleteTablet for tablet a76d543997024a949d505cf045afc6b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:30 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:59172
I20260812 04:01:30.424142 19066 tablet_service.cc:1514] Processing DeleteTablet for tablet a76d543997024a949d505cf045afc6b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:30 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:36154
I20260812 04:01:30.424173 19217 tablet_service.cc:1514] Processing DeleteTablet for tablet a76d543997024a949d505cf045afc6b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:30 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:33974
I20260812 04:01:30.424401 19492 tablet_replica.cc:330] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4: stopping tablet replica
I20260812 04:01:30.424518 19492 raft_consensus.cc:2229] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:01:30.424638 19492 raft_consensus.cc:2258] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:30.424765 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 18997
I20260812 04:01:30.425161 19492 ts_tablet_manager.cc:1905] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:01:30.427454 19492 ts_tablet_manager.cc:1918] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:01:30.427560 19492 log.cc:1196] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/a76d543997024a949d505cf045afc6b5
I20260812 04:01:30.427906 19492 ts_tablet_manager.cc:1939] T a76d543997024a949d505cf045afc6b5 P f24d11805557490f80148e24d431e5e4: Deleting consensus metadata
I20260812 04:01:30.428579 18907 catalog_manager.cc:4841] TS f24d11805557490f80148e24d431e5e4 (127.31.43.194:35147): tablet a76d543997024a949d505cf045afc6b5 (table test-table [id=9c06d1a8f7a44778b05245a54605260d]) successfully deleted
I20260812 04:01:30.432637 19491 tablet_replica.cc:330] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc: stopping tablet replica
I20260812 04:01:30.432796 19491 raft_consensus.cc:2229] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 1 LEADER]: Raft consensus shutting down.
I20260812 04:01:30.432924 19491 raft_consensus.cc:2258] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:30.433491 19491 ts_tablet_manager.cc:1905] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:01:30.435732 19491 ts_tablet_manager.cc:1918] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:01:30.435824 19491 log.cc:1196] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/a76d543997024a949d505cf045afc6b5
I20260812 04:01:30.436151 19491 ts_tablet_manager.cc:1939] T a76d543997024a949d505cf045afc6b5 P b19e86096cc14e35b19f56fb8841d0bc: Deleting consensus metadata
I20260812 04:01:30.436849 18910 catalog_manager.cc:4841] TS b19e86096cc14e35b19f56fb8841d0bc (127.31.43.195:43095): tablet a76d543997024a949d505cf045afc6b5 (table test-table [id=9c06d1a8f7a44778b05245a54605260d]) successfully deleted
W20260812 04:01:30.438563 18909 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:40061 (error 108)
I20260812 04:01:30.438905 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 19148
I20260812 04:01:30.441160 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.439069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:40061 (local address 127.0.0.1:36170)
0812 04:01:30.441029 (+ 1960us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:30.441106 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1922}
W20260812 04:01:30.444000 18909 catalog_manager.cc:4584] TS a272b800846543a68d783c83fa3aad84 (127.31.43.193:40061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a76d543997024a949d505cf045afc6b5: Network error: Client connection negotiation failed: client connection to 127.31.43.193:40061: connect: Connection refused (error 111)
I20260812 04:01:30.454804 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 19300
I20260812 04:01:30.462149 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 18891
2026-08-12T04:01:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2427 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T04:01:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:30Z Disabled control of system clock
I20260812 04:01:30.495177 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:42071
--webserver_interface=127.31.43.254
--webserver_port=0
--builtin_ntp_servers=127.31.43.212:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:42071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 04:01:30.578660 19498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:30.579000 19498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:30.579038 19498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:30.580214 19498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:30.580283 19498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:30.580307 19498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:30.580328 19498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:30.580356 19498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:30.581749 19498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:42071
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:42071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:30.582083 19498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:30.582401 19498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:30.587405 19503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:30.587417 19506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:30.587733 19498 server_base.cc:1034] running on GCE node
W20260812 04:01:30.587417 19504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:30.588078 19498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:30.588382 19498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:30.589556 19498 hybrid_clock.cc:648] HybridClock initialized: now 1786507290589529 us; error 55 us; skew 500 ppm
I20260812 04:01:30.591001 19498 webserver.cc:460] Webserver started at http://127.31.43.254:44291/ using document root <none> and password file <none>
I20260812 04:01:30.591259 19498 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:30.591329 19498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:30.591493 19498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:30.592638 19498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "dab51234828947c88049639364338224"
format_stamp: "Formatted at 2026-08-12 04:01:30 on dist-test-slave-8tdl"
server_key: "59627e9c718f5b603461eceb1fff3ca4"
server_key_iv: "e793dbd5a57f5ba1cc95f6585730f5d0"
server_key_version: "encryptionkey@0"
I20260812 04:01:30.593040 19498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "dab51234828947c88049639364338224"
format_stamp: "Formatted at 2026-08-12 04:01:30 on dist-test-slave-8tdl"
server_key: "59627e9c718f5b603461eceb1fff3ca4"
server_key_iv: "e793dbd5a57f5ba1cc95f6585730f5d0"
server_key_version: "encryptionkey@0"
I20260812 04:01:30.594476 19498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:30.595516 19513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:30.595748 19498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:30.595867 19498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "dab51234828947c88049639364338224"
format_stamp: "Formatted at 2026-08-12 04:01:30 on dist-test-slave-8tdl"
server_key: "59627e9c718f5b603461eceb1fff3ca4"
server_key_iv: "e793dbd5a57f5ba1cc95f6585730f5d0"
server_key_version: "encryptionkey@0"
I20260812 04:01:30.595956 19498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:30.604943 19498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:30.605295 19498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:30.605466 19498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:30.609688 19498 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:42071
I20260812 04:01:30.609725 19568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:42071 every 8 connection(s)
I20260812 04:01:30.610173 19498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:30.611501 19569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:30.613929 19569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dab51234828947c88049639364338224: Bootstrap starting.
I20260812 04:01:30.614550 19569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dab51234828947c88049639364338224: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:30.614934 19569 log.cc:824] T 00000000000000000000000000000000 P dab51234828947c88049639364338224: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:30.615736 19569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dab51234828947c88049639364338224: No bootstrap required, opened a new log
I20260812 04:01:30.617473 19569 raft_consensus.cc:348] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab51234828947c88049639364338224" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42071 } }
I20260812 04:01:30.617627 19569 raft_consensus.cc:374] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:30.617652 19569 raft_consensus.cc:729] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dab51234828947c88049639364338224, State: Initialized, Role: FOLLOWER
I20260812 04:01:30.617822 19569 consensus_queue.cc:260] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab51234828947c88049639364338224" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42071 } }
I20260812 04:01:30.617898 19569 raft_consensus.cc:388] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:30.617942 19569 raft_consensus.cc:482] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:30.617992 19569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:30.618571 19569 raft_consensus.cc:504] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab51234828947c88049639364338224" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42071 } }
I20260812 04:01:30.618662 19569 leader_election.cc:302] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dab51234828947c88049639364338224; no voters:
I20260812 04:01:30.618912 19569 leader_election.cc:288] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:30.618980 19572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:30.619210 19572 raft_consensus.cc:686] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [term 1 LEADER]: Becoming Leader. State: Replica: dab51234828947c88049639364338224, State: Running, Role: LEADER
I20260812 04:01:30.619259 19569 sys_catalog.cc:564] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:30.619346 19572 consensus_queue.cc:237] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab51234828947c88049639364338224" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42071 } }
I20260812 04:01:30.619755 19574 sys_catalog.cc:455] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dab51234828947c88049639364338224. Latest consensus state: current_term: 1 leader_uuid: "dab51234828947c88049639364338224" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab51234828947c88049639364338224" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42071 } } }
I20260812 04:01:30.619843 19573 sys_catalog.cc:455] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dab51234828947c88049639364338224" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab51234828947c88049639364338224" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 42071 } } }
I20260812 04:01:30.619984 19573 sys_catalog.cc:458] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:30.620082 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 19498
I20260812 04:01:30.620198 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:30.620432 31919 external_mini_cluster.cc:1428] Setting key 734854b65ba5714a1e4bc6c135d5168e
I20260812 04:01:30.619868 19574 sys_catalog.cc:458] T 00000000000000000000000000000000 P dab51234828947c88049639364338224 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:30.620543 19582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:30.621307 19582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:30.622860 19582 catalog_manager.cc:1347] Generated new cluster ID: 9648a3f49671460e86e08a33e47ea27f
I20260812 04:01:30.622920 19582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:30.631475 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.620944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58728 (local address 127.31.43.254:42071)
0812 04:01:30.626776 (+ 5832us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:30.626785 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:30.626818 (+ 33us) server_negotiation.cc:407] Connection header received
0812 04:01:30.626924 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:30.626930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:30.626988 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:30.627160 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:30.627392 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.628322 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.629277 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.629734 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.629878 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.630652 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:30.630656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:30.630656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:30.630907 (+ 251us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:30.630945 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:30.630957 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:30.631046 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:30.631133 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 04:01:30.631222 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5690,"thread_start_us":90,"threads_started":1}
I20260812 04:01:30.643261 19582 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:30.644021 19582 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:30.648656 19582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dab51234828947c88049639364338224: Generated new TSK 0
I20260812 04:01:30.648855 19582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:30.652447 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42071
--builtin_ntp_servers=127.31.43.212:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 04:01:30.739898 19596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:30.740160 19596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:30.740199 19596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:30.741458 19596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:30.741526 19596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:30.741570 19596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:01:30.743100 19596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:30.743394 19596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:30.743726 19596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 04:01:30.749475 19596 server_base.cc:1034] running on GCE node
W20260812 04:01:30.749333 19602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:30.749330 19601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:30.749397 19604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:30.749881 19596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:30.750159 19596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:30.751327 19596 hybrid_clock.cc:648] HybridClock initialized: now 1786507290751301 us; error 40 us; skew 500 ppm
I20260812 04:01:30.752926 19596 webserver.cc:460] Webserver started at http://127.31.43.193:44269/ using document root <none> and password file <none>
I20260812 04:01:30.753221 19596 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:30.753283 19596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:30.753417 19596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:30.754997 19596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "ddf62a4fe95242efab1c773ac8e58e34"
format_stamp: "Formatted at 2026-08-12 04:01:30 on dist-test-slave-8tdl"
server_key: "86811ea242b6f3ed8a314b88d199a2b8"
server_key_iv: "7f88dcd6ef6dab57955a557b28daedac"
server_key_version: "encryptionkey@0"
I20260812 04:01:30.755511 19596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "ddf62a4fe95242efab1c773ac8e58e34"
format_stamp: "Formatted at 2026-08-12 04:01:30 on dist-test-slave-8tdl"
server_key: "86811ea242b6f3ed8a314b88d199a2b8"
server_key_iv: "7f88dcd6ef6dab57955a557b28daedac"
server_key_version: "encryptionkey@0"
I20260812 04:01:30.757184 19596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:01:30.758111 19612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:30.758351 19596 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:30.758455 19596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "ddf62a4fe95242efab1c773ac8e58e34"
format_stamp: "Formatted at 2026-08-12 04:01:30 on dist-test-slave-8tdl"
server_key: "86811ea242b6f3ed8a314b88d199a2b8"
server_key_iv: "7f88dcd6ef6dab57955a557b28daedac"
server_key_version: "encryptionkey@0"
I20260812 04:01:30.758534 19596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:30.776325 19596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:30.776647 19596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:30.776763 19596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:30.776969 19596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:30.777362 19596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:30.777400 19596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:30.777460 19596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:30.777485 19596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:30.783874 19596 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:46483
I20260812 04:01:30.783915 19747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:46483 every 8 connection(s)
I20260812 04:01:30.784323 19596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 04:01:30.789064 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 19596
I20260812 04:01:30.789213 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 04:01:30.789471 31919 external_mini_cluster.cc:1428] Setting key acab3488689cd9c7a01b61a2fbb38892
I20260812 04:01:30.790366 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.785201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:56187 (local address 127.31.43.254:42071)
0812 04:01:30.785289 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:30.785291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:30.785503 (+ 212us) server_negotiation.cc:407] Connection header received
0812 04:01:30.786117 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:30.786119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:30.786137 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:30.786220 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:30.787070 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.787590 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.788617 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.788865 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.788940 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.789978 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:30.789979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:30.789979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:30.789984 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:30.790002 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:30.790007 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:30.790117 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:30.790242 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 04:01:30.790284 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 04:01:30.790606 19751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.785125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42071 (local address 127.31.43.193:56187)
0812 04:01:30.785376 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:30.785414 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:30.785965 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:30.786313 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:30.786316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:30.786326 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:30.786962 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.786968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.787694 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.787695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.788493 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.788495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.788990 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.788992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.789128 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.789843 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 04:01:30.789843 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:30.789874 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 04:01:30.789900 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:30.790149 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:30.790156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:30.790159 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:01:30.790244 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 04:01:30.790361 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":90,"threads_started":1}
I20260812 04:01:30.790670 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42071
--builtin_ntp_servers=127.31.43.212:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 04:01:30.791474 19748 heartbeater.cc:347] Connected to a master server at 127.31.43.254:42071
I20260812 04:01:30.791610 19748 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:30.791827 19748 heartbeater.cc:511] Master 127.31.43.254:42071 requested a full tablet report, sending...
I20260812 04:01:30.792438 19530 ts_manager.cc:194] Registered new tserver with Master: ddf62a4fe95242efab1c773ac8e58e34 (127.31.43.193:46483)
I20260812 04:01:30.793159 19530 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.43.193:56187
W20260812 04:01:30.877965 19752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:30.878204 19752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:30.878227 19752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:30.879433 19752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:30.879477 19752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:30.879508 19752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:01:30.880884 19752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:30.881135 19752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:30.881446 19752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:30.886863 19761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:30.887006 19752 server_base.cc:1034] running on GCE node
W20260812 04:01:30.886878 19757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:30.886900 19758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:30.887316 19752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:30.887612 19752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:30.888799 19752 hybrid_clock.cc:648] HybridClock initialized: now 1786507290888788 us; error 50 us; skew 500 ppm
I20260812 04:01:30.890161 19752 webserver.cc:460] Webserver started at http://127.31.43.194:34381/ using document root <none> and password file <none>
I20260812 04:01:30.890432 19752 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:30.890532 19752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:30.890800 19752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:30.892073 19752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "ff34589a72a842bb993e97c01cf5843a"
format_stamp: "Formatted at 2026-08-12 04:01:30 on dist-test-slave-8tdl"
server_key: "fb9d1d7b13a9fd494df73fa81d71976b"
server_key_iv: "ff2512db18a3a9e0685c7e7131b5de66"
server_key_version: "encryptionkey@0"
I20260812 04:01:30.892545 19752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "ff34589a72a842bb993e97c01cf5843a"
format_stamp: "Formatted at 2026-08-12 04:01:30 on dist-test-slave-8tdl"
server_key: "fb9d1d7b13a9fd494df73fa81d71976b"
server_key_iv: "ff2512db18a3a9e0685c7e7131b5de66"
server_key_version: "encryptionkey@0"
I20260812 04:01:30.894071 19752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:30.894986 19767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:30.895161 19752 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:30.895283 19752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "ff34589a72a842bb993e97c01cf5843a"
format_stamp: "Formatted at 2026-08-12 04:01:30 on dist-test-slave-8tdl"
server_key: "fb9d1d7b13a9fd494df73fa81d71976b"
server_key_iv: "ff2512db18a3a9e0685c7e7131b5de66"
server_key_version: "encryptionkey@0"
I20260812 04:01:30.895371 19752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:30.921965 19752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:30.922353 19752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:30.922544 19752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:30.922881 19752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:30.923276 19752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:30.923341 19752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:30.923388 19752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:30.923421 19752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:30.930356 19752 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:36297
I20260812 04:01:30.930408 19897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:36297 every 8 connection(s)
I20260812 04:01:30.930836 19752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:01:30.936528 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.931741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:33855 (local address 127.31.43.254:42071)
0812 04:01:30.931828 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:30.931830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:30.932060 (+ 230us) server_negotiation.cc:407] Connection header received
0812 04:01:30.932609 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:30.932610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:30.932622 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:30.932686 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:30.933374 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.933937 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.935024 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.935231 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.935296 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.936218 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:30.936220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:30.936220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:30.936225 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:30.936242 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:30.936246 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:30.936298 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:30.936434 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 04:01:30.936465 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 04:01:30.936820 19900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:30.931682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42071 (local address 127.31.43.194:33855)
0812 04:01:30.931904 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:30.931960 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:30.932486 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:30.932749 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:30.932752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:30.932761 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:30.933288 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.933292 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.934090 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.934091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.934933 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:30.934936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:30.935315 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:30.935316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:30.935438 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:30.936064 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 04:01:30.936065 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:30.936095 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 04:01:30.936121 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:30.936341 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:30.936349 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:30.936352 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:01:30.936424 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 04:01:30.936524 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260812 04:01:30.937475 19898 heartbeater.cc:347] Connected to a master server at 127.31.43.254:42071
I20260812 04:01:30.937614 19898 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:30.937850 19898 heartbeater.cc:511] Master 127.31.43.254:42071 requested a full tablet report, sending...
I20260812 04:01:30.937940 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 19752
I20260812 04:01:30.938045 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 04:01:30.938277 31919 external_mini_cluster.cc:1428] Setting key d1b737513983d76367dd1582375bbd41
I20260812 04:01:30.938390 19530 ts_manager.cc:194] Registered new tserver with Master: ff34589a72a842bb993e97c01cf5843a (127.31.43.194:36297)
I20260812 04:01:30.938892 19530 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.43.194:33855
I20260812 04:01:30.939632 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42071
--builtin_ntp_servers=127.31.43.212:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 04:01:31.025664 19901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:31.025890 19901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:31.025913 19901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:31.027074 19901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:31.027112 19901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:31.027138 19901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:01:31.028615 19901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:42071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:31.028851 19901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:31.029134 19901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:31.034943 19912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:31.034960 19909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:31.034981 19908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:31.035306 19901 server_base.cc:1034] running on GCE node
I20260812 04:01:31.035574 19901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:31.035861 19901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:31.037022 19901 hybrid_clock.cc:648] HybridClock initialized: now 1786507291037004 us; error 33 us; skew 500 ppm
I20260812 04:01:31.038625 19901 webserver.cc:460] Webserver started at http://127.31.43.195:45059/ using document root <none> and password file <none>
I20260812 04:01:31.039072 19901 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:31.039140 19901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:31.039342 19901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:31.040563 19901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "09498bf5813b4c99bf951e5c93d10204"
format_stamp: "Formatted at 2026-08-12 04:01:31 on dist-test-slave-8tdl"
server_key: "159a9ebabf9aa27863e86f9714962d17"
server_key_iv: "38dd65c18a240907de97fd1ceded7a28"
server_key_version: "encryptionkey@0"
I20260812 04:01:31.040984 19901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "09498bf5813b4c99bf951e5c93d10204"
format_stamp: "Formatted at 2026-08-12 04:01:31 on dist-test-slave-8tdl"
server_key: "159a9ebabf9aa27863e86f9714962d17"
server_key_iv: "38dd65c18a240907de97fd1ceded7a28"
server_key_version: "encryptionkey@0"
I20260812 04:01:31.042455 19901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:31.043444 19919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:31.043651 19901 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:31.043774 19901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "09498bf5813b4c99bf951e5c93d10204"
format_stamp: "Formatted at 2026-08-12 04:01:31 on dist-test-slave-8tdl"
server_key: "159a9ebabf9aa27863e86f9714962d17"
server_key_iv: "38dd65c18a240907de97fd1ceded7a28"
server_key_version: "encryptionkey@0"
I20260812 04:01:31.043854 19901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:31.079622 19901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:31.079991 19901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:31.080163 19901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:31.080425 19901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:31.080785 19901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:31.080840 19901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:31.080881 19901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:31.080925 19901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:31.087173 19901 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:44315
I20260812 04:01:31.087236 20046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:44315 every 8 connection(s)
I20260812 04:01:31.087610 19901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 04:01:31.093062 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.088442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:38189 (local address 127.31.43.254:42071)
0812 04:01:31.088539 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:31.088541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:31.088766 (+ 225us) server_negotiation.cc:407] Connection header received
0812 04:01:31.089324 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:31.089326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:31.089345 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:31.089400 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:31.090101 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.090617 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.091647 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.091861 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.091951 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.092760 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:31.092761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:31.092762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:31.092767 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:31.092782 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:31.092787 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:31.092843 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:31.092937 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 04:01:31.092973 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 04:01:31.093407 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.088405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:42071 (local address 127.31.43.195:38189)
0812 04:01:31.088634 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:31.088670 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:31.089204 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:31.089476 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:31.089480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:31.089487 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:31.090000 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:31.090004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.090746 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.090747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:31.091531 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:31.091533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.091967 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.091968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:31.092104 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.092611 (+ 507us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 04:01:31.092611 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:31.092641 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 04:01:31.092664 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:31.092845 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:31.092850 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:31.092853 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:01:31.092940 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 04:01:31.093163 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
I20260812 04:01:31.093966 20047 heartbeater.cc:347] Connected to a master server at 127.31.43.254:42071
I20260812 04:01:31.094071 20047 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:31.094226 20047 heartbeater.cc:511] Master 127.31.43.254:42071 requested a full tablet report, sending...
I20260812 04:01:31.094635 19529 ts_manager.cc:194] Registered new tserver with Master: 09498bf5813b4c99bf951e5c93d10204 (127.31.43.195:44315)
I20260812 04:01:31.095036 19529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.43.195:38189
I20260812 04:01:31.096305 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 19901
I20260812 04:01:31.096398 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 04:01:31.096552 31919 external_mini_cluster.cc:1428] Setting key 3fb0b49095b0885249c245bd3ebc073d
I20260812 04:01:31.097427 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 04:01:31.102097 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.098981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58732 (local address 127.31.43.254:42071)
0812 04:01:31.099023 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:31.099025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:31.099162 (+ 137us) server_negotiation.cc:407] Connection header received
0812 04:01:31.099243 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:31.099244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:31.099254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:31.099330 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:31.099522 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.100000 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.100840 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.101039 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.101121 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.101828 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:31.101829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:31.101830 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:31.101835 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:31.101850 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:31.101854 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:31.101930 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:31.101984 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 04:01:31.102008 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 04:01:31.104072 19529 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 04:01:31.105650 19529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 04:01:31.112027 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.108100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52814 (local address 127.31.43.195:44315)
0812 04:01:31.108510 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:31.108514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:31.108521 (+ 7us) server_negotiation.cc:407] Connection header received
0812 04:01:31.108550 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:31.108551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:31.108563 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:31.108650 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:31.108868 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.109689 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.110895 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.111620 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.111720 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.111813 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:31.111901 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 04:01:31.111939 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":87,"threads_started":1}
I20260812 04:01:31.112020 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.107936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:44315 (local address 127.0.0.1:52814)
0812 04:01:31.108272 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:31.108305 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:31.108359 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:31.108721 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:31.108724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:31.108726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:31.108792 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:31.108794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.109770 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.109771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:31.110797 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:31.110800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.111721 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.111722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:31.111798 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.111811 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 04:01:31.111887 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 04:01:31.111939 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":61,"threads_started":1}
I20260812 04:01:31.112568 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.107905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:46483 (local address 127.0.0.1:37238)
0812 04:01:31.108249 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:31.108290 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:31.108360 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:31.108939 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:31.108940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:31.108942 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:31.108998 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:31.109000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.110327 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.110328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:31.111292 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:31.111295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.112312 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.112313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:31.112381 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.112386 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 04:01:31.112433 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 04:01:31.112476 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":80,"threads_started":1}
I20260812 04:01:31.112668 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.108272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44134 (local address 127.31.43.194:36297)
0812 04:01:31.108484 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:31.108489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:31.108506 (+ 17us) server_negotiation.cc:407] Connection header received
0812 04:01:31.108566 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:31.108568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:31.108584 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:31.109139 (+ 555us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:31.109272 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.110321 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.111595 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.112316 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.112383 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.112463 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:31.112534 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 04:01:31.112572 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":103,"threads_started":1}
I20260812 04:01:31.112821 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.108030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37238 (local address 127.31.43.193:46483)
0812 04:01:31.108726 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:31.108730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:31.108740 (+ 10us) server_negotiation.cc:407] Connection header received
0812 04:01:31.108777 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:31.108779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:31.108795 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:31.108869 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:31.109099 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.110225 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.111392 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.112107 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.112184 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.112268 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:31.112693 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 04:01:31.112744 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":108,"threads_started":1}
I20260812 04:01:31.113049 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.108152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:36297 (local address 127.0.0.1:44134)
0812 04:01:31.108279 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:31.108293 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:31.108359 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:31.109153 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:31.109153 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:31.109156 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:31.109195 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:31.109196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.110407 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.110408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:31.111503 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:31.111506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.112388 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.112389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:31.112451 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.112455 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:01:31.112522 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 04:01:31.112564 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260812 04:01:31.112953 19974 tablet_service.cc:1467] Processing CreateTablet for tablet c86ba56779794ebfb59589b809450245 (DEFAULT_TABLE table=test-table [id=a9b29b24ab894eceb1f86a3cc7c24c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:31.113356 19974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c86ba56779794ebfb59589b809450245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:31.115495 19672 tablet_service.cc:1467] Processing CreateTablet for tablet c86ba56779794ebfb59589b809450245 (DEFAULT_TABLE table=test-table [id=a9b29b24ab894eceb1f86a3cc7c24c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:31.115490 19825 tablet_service.cc:1467] Processing CreateTablet for tablet c86ba56779794ebfb59589b809450245 (DEFAULT_TABLE table=test-table [id=a9b29b24ab894eceb1f86a3cc7c24c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:31.115856 19672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c86ba56779794ebfb59589b809450245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:31.115856 19825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c86ba56779794ebfb59589b809450245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:31.116371 20064 tablet_bootstrap.cc:492] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204: Bootstrap starting.
I20260812 04:01:31.117040 20064 tablet_bootstrap.cc:654] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:31.117340 20064 log.cc:824] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:31.118036 20064 tablet_bootstrap.cc:492] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204: No bootstrap required, opened a new log
I20260812 04:01:31.118122 20064 ts_tablet_manager.cc:1397] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 04:01:31.119277 20067 tablet_bootstrap.cc:492] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34: Bootstrap starting.
I20260812 04:01:31.120162 20067 tablet_bootstrap.cc:654] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:31.120496 20067 log.cc:824] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:31.120841 20064 raft_consensus.cc:348] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf62a4fe95242efab1c773ac8e58e34" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 46483 } } peers { permanent_uuid: "ff34589a72a842bb993e97c01cf5843a" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36297 } } peers { permanent_uuid: "09498bf5813b4c99bf951e5c93d10204" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44315 } }
I20260812 04:01:31.120977 20064 raft_consensus.cc:374] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:31.121048 20064 raft_consensus.cc:729] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09498bf5813b4c99bf951e5c93d10204, State: Initialized, Role: FOLLOWER
I20260812 04:01:31.121170 20064 consensus_queue.cc:260] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf62a4fe95242efab1c773ac8e58e34" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 46483 } } peers { permanent_uuid: "ff34589a72a842bb993e97c01cf5843a" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36297 } } peers { permanent_uuid: "09498bf5813b4c99bf951e5c93d10204" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44315 } }
I20260812 04:01:31.121419 20067 tablet_bootstrap.cc:492] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34: No bootstrap required, opened a new log
I20260812 04:01:31.121449 20064 ts_tablet_manager.cc:1428] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 04:01:31.121502 20067 ts_tablet_manager.cc:1397] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 04:01:31.121511 20047 heartbeater.cc:503] Master 127.31.43.254:42071 was elected leader, sending a full tablet report...
I20260812 04:01:31.121665 20068 tablet_bootstrap.cc:492] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a: Bootstrap starting.
I20260812 04:01:31.122397 20068 tablet_bootstrap.cc:654] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:31.122761 20068 log.cc:824] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:31.123665 20068 tablet_bootstrap.cc:492] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a: No bootstrap required, opened a new log
I20260812 04:01:31.123745 20068 ts_tablet_manager.cc:1397] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 04:01:31.124300 20067 raft_consensus.cc:348] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf62a4fe95242efab1c773ac8e58e34" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 46483 } } peers { permanent_uuid: "ff34589a72a842bb993e97c01cf5843a" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36297 } } peers { permanent_uuid: "09498bf5813b4c99bf951e5c93d10204" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44315 } }
I20260812 04:01:31.124521 20067 raft_consensus.cc:374] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:31.124564 20067 raft_consensus.cc:729] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddf62a4fe95242efab1c773ac8e58e34, State: Initialized, Role: FOLLOWER
I20260812 04:01:31.124683 20067 consensus_queue.cc:260] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf62a4fe95242efab1c773ac8e58e34" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 46483 } } peers { permanent_uuid: "ff34589a72a842bb993e97c01cf5843a" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36297 } } peers { permanent_uuid: "09498bf5813b4c99bf951e5c93d10204" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44315 } }
I20260812 04:01:31.124958 20067 ts_tablet_manager.cc:1428] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 04:01:31.125005 19748 heartbeater.cc:503] Master 127.31.43.254:42071 was elected leader, sending a full tablet report...
I20260812 04:01:31.126331 20068 raft_consensus.cc:348] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf62a4fe95242efab1c773ac8e58e34" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 46483 } } peers { permanent_uuid: "ff34589a72a842bb993e97c01cf5843a" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36297 } } peers { permanent_uuid: "09498bf5813b4c99bf951e5c93d10204" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44315 } }
I20260812 04:01:31.126470 20068 raft_consensus.cc:374] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:31.126493 20068 raft_consensus.cc:729] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff34589a72a842bb993e97c01cf5843a, State: Initialized, Role: FOLLOWER
I20260812 04:01:31.126627 20068 consensus_queue.cc:260] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf62a4fe95242efab1c773ac8e58e34" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 46483 } } peers { permanent_uuid: "ff34589a72a842bb993e97c01cf5843a" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36297 } } peers { permanent_uuid: "09498bf5813b4c99bf951e5c93d10204" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44315 } }
I20260812 04:01:31.126914 20068 ts_tablet_manager.cc:1428] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 04:01:31.126973 19898 heartbeater.cc:503] Master 127.31.43.254:42071 was elected leader, sending a full tablet report...
I20260812 04:01:31.167914 20070 raft_consensus.cc:482] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:01:31.168061 20070 raft_consensus.cc:504] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf62a4fe95242efab1c773ac8e58e34" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 46483 } } peers { permanent_uuid: "ff34589a72a842bb993e97c01cf5843a" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36297 } } peers { permanent_uuid: "09498bf5813b4c99bf951e5c93d10204" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44315 } }
I20260812 04:01:31.168489 20070 leader_election.cc:288] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddf62a4fe95242efab1c773ac8e58e34 (127.31.43.193:46483), ff34589a72a842bb993e97c01cf5843a (127.31.43.194:36297)
I20260812 04:01:31.171623 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.168731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:46483 (local address 127.31.43.195:49627)
0812 04:01:31.168798 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:31.168813 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:31.168889 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:31.169092 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:31.169093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:31.169096 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:31.169185 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:31.169187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.169979 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.169980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:31.170801 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:31.170803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.171356 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.171357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:31.171429 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.171432 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:01:31.171500 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 04:01:31.171546 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 04:01:31.171623 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.168824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:49627 (local address 127.31.43.193:46483)
0812 04:01:31.168906 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:31.168908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:31.168920 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:01:31.168956 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:31.168958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:31.168972 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:31.169266 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:31.169321 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.169878 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.170892 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.171259 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.171344 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.171353 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:31.171508 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 04:01:31.171548 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 04:01:31.171976 19694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86ba56779794ebfb59589b809450245" candidate_uuid: "09498bf5813b4c99bf951e5c93d10204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddf62a4fe95242efab1c773ac8e58e34" is_pre_election: true
I20260812 04:01:31.172106 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.168877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:36297 (local address 127.31.43.195:45003)
0812 04:01:31.169086 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:31.169097 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:31.169155 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:31.169318 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:31.169319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:31.169321 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:31.169415 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:31.169417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.170131 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.170132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:31.170980 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:31.170982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.171612 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.171613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:31.171837 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.171843 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 04:01:31.171951 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 04:01:31.171995 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":139,"threads_started":1}
I20260812 04:01:31.172094 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.169027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:45003 (local address 127.31.43.194:36297)
0812 04:01:31.169111 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:31.169113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:31.169147 (+ 34us) server_negotiation.cc:407] Connection header received
0812 04:01:31.169222 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:31.169223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:31.169236 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:31.169293 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:31.169524 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.170028 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.171104 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.171504 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.171602 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.171610 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:31.171969 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 04:01:31.172014 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 04:01:31.172151 19694 raft_consensus.cc:2454] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09498bf5813b4c99bf951e5c93d10204 in term 0.
I20260812 04:01:31.172418 19923 leader_election.cc:302] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09498bf5813b4c99bf951e5c93d10204, ddf62a4fe95242efab1c773ac8e58e34; no voters:
I20260812 04:01:31.172444 19849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86ba56779794ebfb59589b809450245" candidate_uuid: "09498bf5813b4c99bf951e5c93d10204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff34589a72a842bb993e97c01cf5843a" is_pre_election: true
I20260812 04:01:31.172588 19849 raft_consensus.cc:2454] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09498bf5813b4c99bf951e5c93d10204 in term 0.
I20260812 04:01:31.172601 20070 raft_consensus.cc:2781] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 04:01:31.172683 20070 raft_consensus.cc:482] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 04:01:31.172719 20070 raft_consensus.cc:3037] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:31.173473 20070 raft_consensus.cc:504] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf62a4fe95242efab1c773ac8e58e34" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 46483 } } peers { permanent_uuid: "ff34589a72a842bb993e97c01cf5843a" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36297 } } peers { permanent_uuid: "09498bf5813b4c99bf951e5c93d10204" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44315 } }
I20260812 04:01:31.173627 20070 leader_election.cc:288] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [CANDIDATE]: Term 1 election: Requested vote from peers ddf62a4fe95242efab1c773ac8e58e34 (127.31.43.193:46483), ff34589a72a842bb993e97c01cf5843a (127.31.43.194:36297)
I20260812 04:01:31.173854 19694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86ba56779794ebfb59589b809450245" candidate_uuid: "09498bf5813b4c99bf951e5c93d10204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddf62a4fe95242efab1c773ac8e58e34"
I20260812 04:01:31.173853 19849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86ba56779794ebfb59589b809450245" candidate_uuid: "09498bf5813b4c99bf951e5c93d10204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff34589a72a842bb993e97c01cf5843a"
I20260812 04:01:31.173949 19849 raft_consensus.cc:3037] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:31.173949 19694 raft_consensus.cc:3037] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:31.175055 19849 raft_consensus.cc:2454] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09498bf5813b4c99bf951e5c93d10204 in term 1.
I20260812 04:01:31.175107 19694 raft_consensus.cc:2454] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09498bf5813b4c99bf951e5c93d10204 in term 1.
I20260812 04:01:31.175276 19923 leader_election.cc:302] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09498bf5813b4c99bf951e5c93d10204, ff34589a72a842bb993e97c01cf5843a; no voters:
I20260812 04:01:31.175377 20070 raft_consensus.cc:2781] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:31.175552 20070 raft_consensus.cc:686] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 1 LEADER]: Becoming Leader. State: Replica: 09498bf5813b4c99bf951e5c93d10204, State: Running, Role: LEADER
I20260812 04:01:31.175671 20070 consensus_queue.cc:237] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf62a4fe95242efab1c773ac8e58e34" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 46483 } } peers { permanent_uuid: "ff34589a72a842bb993e97c01cf5843a" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36297 } } peers { permanent_uuid: "09498bf5813b4c99bf951e5c93d10204" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44315 } }
I20260812 04:01:31.176671 19529 catalog_manager.cc:5495] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09498bf5813b4c99bf951e5c93d10204 (127.31.43.195). New cstate: current_term: 1 leader_uuid: "09498bf5813b4c99bf951e5c93d10204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf62a4fe95242efab1c773ac8e58e34" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 46483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff34589a72a842bb993e97c01cf5843a" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09498bf5813b4c99bf951e5c93d10204" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44315 } health_report { overall_health: HEALTHY } } }
W20260812 04:01:31.181412 19899 tablet.cc:2369] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:01:31.189611 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.186617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52822 (local address 127.31.43.195:44315)
0812 04:01:31.186708 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:31.186710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:31.186720 (+ 10us) server_negotiation.cc:407] Connection header received
0812 04:01:31.186790 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:31.186791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:31.186814 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:31.186948 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:01:31.187143 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.187688 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.188721 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:31.188944 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:31.189041 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:31.189231 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:01:31.189341 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:01:31.189440 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:31.189482 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 04:01:31.189501 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 04:01:31.191639 19694 raft_consensus.cc:1261] T c86ba56779794ebfb59589b809450245 P ddf62a4fe95242efab1c773ac8e58e34 [term 1 FOLLOWER]: Refusing update from remote peer 09498bf5813b4c99bf951e5c93d10204: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:01:31.191668 19849 raft_consensus.cc:1261] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a [term 1 FOLLOWER]: Refusing update from remote peer 09498bf5813b4c99bf951e5c93d10204: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:01:31.192116 20070 consensus_queue.cc:1035] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddf62a4fe95242efab1c773ac8e58e34" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 46483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:01:31.192284 20078 consensus_queue.cc:1035] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff34589a72a842bb993e97c01cf5843a" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:01:31.195161 20083 mvcc.cc:205] Tried to move back new op lower bound from 7317533864718209024 to 7317533864656441344. Current Snapshot: MvccSnapshot[applied={T|T < 7317533864718209024}]
I20260812 04:01:31.195785 20082 mvcc.cc:205] Tried to move back new op lower bound from 7317533864718209024 to 7317533864656441344. Current Snapshot: MvccSnapshot[applied={T|T < 7317533864718209024 or (T in {7317533864718209024})}]
I20260812 04:01:31.196884 19530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:01:31.196971 19530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:01:31.197886 19530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dab51234828947c88049639364338224: Sending DeleteTablet for 3 replicas of tablet c86ba56779794ebfb59589b809450245
I20260812 04:01:31.198238 19974 tablet_service.cc:1514] Processing DeleteTablet for tablet c86ba56779794ebfb59589b809450245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:31 UTC) from {username='slave'} at 127.0.0.1:52814
I20260812 04:01:31.198328 19825 tablet_service.cc:1514] Processing DeleteTablet for tablet c86ba56779794ebfb59589b809450245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:31 UTC) from {username='slave'} at 127.0.0.1:44134
I20260812 04:01:31.198329 19672 tablet_service.cc:1514] Processing DeleteTablet for tablet c86ba56779794ebfb59589b809450245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:31 UTC) from {username='slave'} at 127.0.0.1:37238
I20260812 04:01:31.198472 20092 tablet_replica.cc:330] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204: stopping tablet replica
I20260812 04:01:31.198572 20092 raft_consensus.cc:2229] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 1 LEADER]: Raft consensus shutting down.
I20260812 04:01:31.198668 20092 raft_consensus.cc:2258] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:31.198730 20093 tablet_replica.cc:330] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a: stopping tablet replica
I20260812 04:01:31.198917 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 19596
I20260812 04:01:31.199218 20092 ts_tablet_manager.cc:1905] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:01:31.201465 20092 ts_tablet_manager.cc:1918] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:01:31.201563 20092 log.cc:1196] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/c86ba56779794ebfb59589b809450245
I20260812 04:01:31.199043 20093 raft_consensus.cc:2229] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:01:31.201905 20092 ts_tablet_manager.cc:1939] T c86ba56779794ebfb59589b809450245 P 09498bf5813b4c99bf951e5c93d10204: Deleting consensus metadata
I20260812 04:01:31.202071 20093 raft_consensus.cc:2258] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:31.202406 19516 catalog_manager.cc:4841] TS 09498bf5813b4c99bf951e5c93d10204 (127.31.43.195:44315): tablet c86ba56779794ebfb59589b809450245 (table test-table [id=a9b29b24ab894eceb1f86a3cc7c24c1a]) successfully deleted
I20260812 04:01:31.203034 20093 ts_tablet_manager.cc:1905] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:01:31.204696 20093 ts_tablet_manager.cc:1918] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:01:31.204782 20093 log.cc:1196] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/c86ba56779794ebfb59589b809450245
I20260812 04:01:31.205056 20093 ts_tablet_manager.cc:1939] T c86ba56779794ebfb59589b809450245 P ff34589a72a842bb993e97c01cf5843a: Deleting consensus metadata
I20260812 04:01:31.205791 19517 catalog_manager.cc:4841] TS ff34589a72a842bb993e97c01cf5843a (127.31.43.194:36297): tablet c86ba56779794ebfb59589b809450245 (table test-table [id=a9b29b24ab894eceb1f86a3cc7c24c1a]) successfully deleted
W20260812 04:01:31.210206 19517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:46483 (error 108)
I20260812 04:01:31.210440 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 19752
I20260812 04:01:31.211575 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:31.210608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:46483 (local address 127.0.0.1:37246)
0812 04:01:31.211478 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:31.211534 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":847}
W20260812 04:01:31.211696 19517 catalog_manager.cc:4584] TS ddf62a4fe95242efab1c773ac8e58e34 (127.31.43.193:46483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c86ba56779794ebfb59589b809450245: Network error: Client connection negotiation failed: client connection to 127.31.43.193:46483: connect: Connection refused (error 111)
I20260812 04:01:31.222584 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 19901
I20260812 04:01:31.230505 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 19498
2026-08-12T04:01:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (766 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:31 dist-test-slave-8tdl krb5kdc[20099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:31 dist-test-slave-8tdl krb5kdc[20099](info): set up 2 sockets
Aug 12 04:01:31 dist-test-slave-8tdl krb5kdc[20099](info): commencing operation
krb5kdc: starting...
W20260812 04:01:32.437755 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:32 dist-test-slave-8tdl krb5kdc[20099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:01:32.566184 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40867
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:40867
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:32.652940 20113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:32.653189 20113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:32.653228 20113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:32.654448 20113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:32.654521 20113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:32.654546 20113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:32.654575 20113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:32.654608 20113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:32.656293 20113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:40867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:32.656596 20113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:32.656850 20113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:32.661984 20121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:32.661984 20118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:32.662014 20119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:32.662556 20113 server_base.cc:1034] running on GCE node
I20260812 04:01:32.662835 20113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:32.663117 20113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:32.664268 20113 hybrid_clock.cc:648] HybridClock initialized: now 1786507292664253 us; error 27 us; skew 500 ppm
Aug 12 04:01:32 dist-test-slave-8tdl krb5kdc[20099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507292, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:32.667029 20113 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:01:32.667501 20113 webserver.cc:460] Webserver started at http://127.31.43.254:40725/ using document root <none> and password file <none>
I20260812 04:01:32.667757 20113 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:32.667807 20113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:32.667908 20113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:32.669096 20113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "fd3c838368594413a2defeeca4a8ac5d"
format_stamp: "Formatted at 2026-08-12 04:01:32 on dist-test-slave-8tdl"
server_key: "ca32a9958823636dadbdac62ceeffc98"
server_key_iv: "34d15f6044ddc5559737f646131514cf"
server_key_version: "encryptionkey@0"
I20260812 04:01:32.669448 20113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "fd3c838368594413a2defeeca4a8ac5d"
format_stamp: "Formatted at 2026-08-12 04:01:32 on dist-test-slave-8tdl"
server_key: "ca32a9958823636dadbdac62ceeffc98"
server_key_iv: "34d15f6044ddc5559737f646131514cf"
server_key_version: "encryptionkey@0"
I20260812 04:01:32.670967 20113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:32.671766 20128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:32.671957 20113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:32.672055 20113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "fd3c838368594413a2defeeca4a8ac5d"
format_stamp: "Formatted at 2026-08-12 04:01:32 on dist-test-slave-8tdl"
server_key: "ca32a9958823636dadbdac62ceeffc98"
server_key_iv: "34d15f6044ddc5559737f646131514cf"
server_key_version: "encryptionkey@0"
I20260812 04:01:32.672117 20113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:32.694538 20113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:32.696646 20113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:32.696766 20113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:32.700950 20113 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:40867
I20260812 04:01:32.701068 20186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:40867 every 8 connection(s)
I20260812 04:01:32.701404 20113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:32.702243 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 20113
I20260812 04:01:32.702383 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:32.702548 20187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:32.702612 31919 external_mini_cluster.cc:1428] Setting key e01883bfa209494787978648e4c5d6b2
I20260812 04:01:32.705600 20187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d: Bootstrap starting.
I20260812 04:01:32.706278 20187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:32.706599 20187 log.cc:824] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:32.707314 20187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d: No bootstrap required, opened a new log
Aug 12 04:01:32 dist-test-slave-8tdl krb5kdc[20099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:32.709805 20187 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd3c838368594413a2defeeca4a8ac5d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40867 } }
I20260812 04:01:32.709978 20187 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:32.710076 20187 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd3c838368594413a2defeeca4a8ac5d, State: Initialized, Role: FOLLOWER
I20260812 04:01:32.710260 20187 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd3c838368594413a2defeeca4a8ac5d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40867 } }
I20260812 04:01:32.710330 20187 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:32.710377 20187 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:32.710428 20187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:32.711036 20187 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd3c838368594413a2defeeca4a8ac5d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40867 } }
I20260812 04:01:32.711130 20187 leader_election.cc:302] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd3c838368594413a2defeeca4a8ac5d; no voters:
I20260812 04:01:32.711311 20187 leader_election.cc:288] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:32.711346 20192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:32.711540 20192 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [term 1 LEADER]: Becoming Leader. State: Replica: fd3c838368594413a2defeeca4a8ac5d, State: Running, Role: LEADER
I20260812 04:01:32.711608 20187 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:32.711728 20192 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd3c838368594413a2defeeca4a8ac5d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40867 } }
I20260812 04:01:32.712121 20192 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd3c838368594413a2defeeca4a8ac5d. Latest consensus state: current_term: 1 leader_uuid: "fd3c838368594413a2defeeca4a8ac5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd3c838368594413a2defeeca4a8ac5d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40867 } } }
I20260812 04:01:32.712292 20192 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:32.712801 20201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:32.713194 20194 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd3c838368594413a2defeeca4a8ac5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd3c838368594413a2defeeca4a8ac5d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40867 } } }
I20260812 04:01:32.713362 20194 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:32.713538 20201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:32.713785 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:32.703180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45944 (local address 127.31.43.254:40867)
0812 04:01:32.703396 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:32.703404 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:32.703430 (+ 26us) server_negotiation.cc:407] Connection header received
0812 04:01:32.703846 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:32.703853 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:32.703900 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:32.704090 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:32.704723 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:32.705601 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:32.706356 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:32.706835 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:32.706955 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:32.709637 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:32.709640 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:32.709641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:32.709651 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:32.711714 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:32.712135 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:32.712139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:32.712139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:32.712245 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:32.712471 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:32.712472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:32.712473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:32.712813 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:32.712971 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:32.713428 (+ 457us) server_negotiation.cc:299] Negotiation successful
0812 04:01:32.713579 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":100,"threads_started":1}
I20260812 04:01:32.715109 20201 catalog_manager.cc:1347] Generated new cluster ID: 66da41aa64c94f61be6615d3863eb02a
I20260812 04:01:32.715170 20201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:32.721416 20201 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:32.721938 20201 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:32.727123 20201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d: Generated new TSK 0
I20260812 04:01:32.727315 20201 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 04:01:32.764135 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40867
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:32.866906 20215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:32.867156 20215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:32.867194 20215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:32.868408 20215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:32.868476 20215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:32.868522 20215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:01:32.870097 20215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:32.870380 20215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:32.870642 20215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:32.875957 20221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:32.875957 20220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:32.875962 20223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:32.876318 20215 server_base.cc:1034] running on GCE node
I20260812 04:01:32.876677 20215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:32.876950 20215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:32.878123 20215 hybrid_clock.cc:648] HybridClock initialized: now 1786507292878110 us; error 30 us; skew 500 ppm
Aug 12 04:01:32 dist-test-slave-8tdl krb5kdc[20099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507292, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:32.880822 20215 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 04:01:32.881304 20215 webserver.cc:460] Webserver started at http://127.31.43.193:37821/ using document root <none> and password file <none>
I20260812 04:01:32.881578 20215 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:32.881650 20215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:32.881803 20215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:32.883042 20215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9"
format_stamp: "Formatted at 2026-08-12 04:01:32 on dist-test-slave-8tdl"
server_key: "3c6168cdc1e5107317340a91e4e644a1"
server_key_iv: "bb81affd9d423960261ffb88061e800e"
server_key_version: "encryptionkey@0"
I20260812 04:01:32.883428 20215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9"
format_stamp: "Formatted at 2026-08-12 04:01:32 on dist-test-slave-8tdl"
server_key: "3c6168cdc1e5107317340a91e4e644a1"
server_key_iv: "bb81affd9d423960261ffb88061e800e"
server_key_version: "encryptionkey@0"
I20260812 04:01:32.884892 20215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:01:32.885702 20230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:32.885921 20215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:32.886034 20215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9"
format_stamp: "Formatted at 2026-08-12 04:01:32 on dist-test-slave-8tdl"
server_key: "3c6168cdc1e5107317340a91e4e644a1"
server_key_iv: "bb81affd9d423960261ffb88061e800e"
server_key_version: "encryptionkey@0"
I20260812 04:01:32.886114 20215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:32.897162 20215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:32.899178 20215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:32.899356 20215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:32.899611 20215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:32.899945 20215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:32.900000 20215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:32.900035 20215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:32.900064 20215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:32.906131 20215 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:33535
I20260812 04:01:32.906184 20351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:33535 every 8 connection(s)
I20260812 04:01:32.906505 20215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 04:01:32.911196 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 20215
I20260812 04:01:32.911329 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 04:01:32.911602 31919 external_mini_cluster.cc:1428] Setting key 164b42e7ebcf3a593d1e20bbcecc6e8b
Aug 12 04:01:32 dist-test-slave-8tdl krb5kdc[20099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507292, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:32.917171 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:32.907281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:57387 (local address 127.31.43.254:40867)
0812 04:01:32.907361 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:32.907362 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:32.907596 (+ 234us) server_negotiation.cc:407] Connection header received
0812 04:01:32.908122 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:32.908123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:32.908133 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:32.908194 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:32.909165 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:32.909664 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:32.910648 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:32.910935 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:32.911021 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:32.913922 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:32.913924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:32.913924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:32.913931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:32.915699 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:32.916139 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:32.916141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:32.916142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:32.916210 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:32.916452 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:32.916453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:32.916453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:32.916603 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:32.916677 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:32.916916 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 04:01:32.917015 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 04:01:32.917193 20354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:32.907264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40867 (local address 127.31.43.193:57387)
0812 04:01:32.907451 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:32.907494 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:32.907993 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:32.908274 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:32.908278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:32.908874 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:32.909074 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:32.909079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:32.909747 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:32.909747 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:32.910552 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:32.910554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:32.911036 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:32.911037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:32.911156 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:32.911795 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:01:32.911808 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:32.913821 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:32.915800 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:32.915807 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:32.915809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:32.916032 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:32.916282 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:32.916283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:32.916283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:32.916386 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:32.916669 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:32.916673 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:32.916789 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 04:01:32.916870 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 04:01:32.916996 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":115,"threads_started":1}
I20260812 04:01:32.917732 20352 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40867
I20260812 04:01:32.917847 20352 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:32.918032 20352 heartbeater.cc:511] Master 127.31.43.254:40867 requested a full tablet report, sending...
I20260812 04:01:32.918597 20146 ts_manager.cc:194] Registered new tserver with Master: cfc6ebcddd4d430aaa6b166fee43bfa9 (127.31.43.193:33535)
I20260812 04:01:32.919238 20146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:57387
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 04:01:32.942312 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40867
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:33.028537 20357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:33.028755 20357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:33.028780 20357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:33.029879 20357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:33.029915 20357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:33.029942 20357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:01:33.031371 20357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:33.031571 20357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:33.031754 20357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:33.037088 20363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:33.037177 20364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:33.037108 20367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:33.037302 20357 server_base.cc:1034] running on GCE node
I20260812 04:01:33.037516 20357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:33.037837 20357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:33.039005 20357 hybrid_clock.cc:648] HybridClock initialized: now 1786507293039001 us; error 35 us; skew 500 ppm
Aug 12 04:01:33 dist-test-slave-8tdl krb5kdc[20099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507293, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:33.041698 20357 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 04:01:33.042173 20357 webserver.cc:460] Webserver started at http://127.31.43.194:33101/ using document root <none> and password file <none>
I20260812 04:01:33.042392 20357 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:33.042434 20357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:33.042526 20357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:33.043632 20357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "c34c1f3c5fec40cd833b4eb41a73af04"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "554aa60ab9cc4b59df353db36ba40cf8"
server_key_iv: "80d9af875aad0a5fddc9774f3838c0a9"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.043942 20357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "c34c1f3c5fec40cd833b4eb41a73af04"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "554aa60ab9cc4b59df353db36ba40cf8"
server_key_iv: "80d9af875aad0a5fddc9774f3838c0a9"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.045275 20357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:01:33.046064 20374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.046281 20357 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:33.046368 20357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "c34c1f3c5fec40cd833b4eb41a73af04"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "554aa60ab9cc4b59df353db36ba40cf8"
server_key_iv: "80d9af875aad0a5fddc9774f3838c0a9"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.046427 20357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:33.057030 20357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:33.059501 20357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:33.059674 20357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:33.059912 20357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:33.060222 20357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:33.060256 20357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.060278 20357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:33.060325 20357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.066053 20357 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:46421
I20260812 04:01:33.066151 20500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:46421 every 8 connection(s)
I20260812 04:01:33.066459 20357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:01:33.068348 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 20357
I20260812 04:01:33.068452 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 04:01:33.068658 31919 external_mini_cluster.cc:1428] Setting key 7f608c2093e66173f51f1799418e26d2
Aug 12 04:01:33 dist-test-slave-8tdl krb5kdc[20099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507293, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:33.077013 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.067243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:44487 (local address 127.31.43.254:40867)
0812 04:01:33.067290 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.067292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.067509 (+ 217us) server_negotiation.cc:407] Connection header received
0812 04:01:33.068040 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.068041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.068053 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.068156 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:33.069434 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.070155 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.071275 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.071540 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.071602 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.073908 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:33.073910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:33.073911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:33.073921 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:33.075516 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:33.075984 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:33.075987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:33.075987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:33.076057 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:33.076320 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:33.076321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:33.076322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:33.076443 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:33.076525 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.076821 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.076918 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 04:01:33.077220 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.067212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40867 (local address 127.31.43.194:44487)
0812 04:01:33.067375 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.067417 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.067921 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.068228 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.068232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.069049 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:33.069317 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.069325 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.070261 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.070262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.071177 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.071180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.071636 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.071637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.071790 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.072407 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:01:33.072416 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:33.073804 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:33.075625 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:33.075634 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:33.075636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:33.075909 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:33.076135 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:33.076136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:33.076136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:33.076252 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:33.076516 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:33.076523 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:33.076679 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.076787 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.076959 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260812 04:01:33.078127 20501 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40867
I20260812 04:01:33.078218 20501 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:33.078424 20501 heartbeater.cc:511] Master 127.31.43.254:40867 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
I20260812 04:01:33.078905 20146 ts_manager.cc:194] Registered new tserver with Master: c34c1f3c5fec40cd833b4eb41a73af04 (127.31.43.194:46421)
I20260812 04:01:33.079304 20146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:44487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 04:01:33.097585 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40867
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:33.183916 20506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:33.184139 20506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:33.184165 20506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:33.185287 20506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:33.185326 20506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:33.185352 20506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:01:33.187026 20506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:40867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:33.187297 20506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:33.187558 20506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:33.192728 20511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:33.192786 20512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:33.192909 20506 server_base.cc:1034] running on GCE node
W20260812 04:01:33.192798 20514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:33.193213 20506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:33.193507 20506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:33.194726 20506 hybrid_clock.cc:648] HybridClock initialized: now 1786507293194702 us; error 34 us; skew 500 ppm
Aug 12 04:01:33 dist-test-slave-8tdl krb5kdc[20099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507293, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:33.197192 20506 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 04:01:33.197582 20506 webserver.cc:460] Webserver started at http://127.31.43.195:35573/ using document root <none> and password file <none>
I20260812 04:01:33.197849 20506 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:33.197896 20506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:33.197992 20506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:33.199093 20506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "2b8cf6fd792a443989b3d57a9abe6e8b"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "117b963c536683ff9c381d71ca6b1ac7"
server_key_iv: "7891e05740e14821b9b46f409aa11832"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.199407 20506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "2b8cf6fd792a443989b3d57a9abe6e8b"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "117b963c536683ff9c381d71ca6b1ac7"
server_key_iv: "7891e05740e14821b9b46f409aa11832"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.200706 20506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:01:33.201385 20522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.201591 20506 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 04:01:33.201673 20506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "2b8cf6fd792a443989b3d57a9abe6e8b"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "117b963c536683ff9c381d71ca6b1ac7"
server_key_iv: "7891e05740e14821b9b46f409aa11832"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.201725 20506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:33.223053 20506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:33.225461 20506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:33.225593 20506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:33.225796 20506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:33.226145 20506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:33.226184 20506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.226209 20506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:33.226222 20506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.232327 20506 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:46609
I20260812 04:01:33.232405 20650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:46609 every 8 connection(s)
I20260812 04:01:33.232760 20506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 04:01:33.233654 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 20506
I20260812 04:01:33.233762 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 04:01:33.233961 31919 external_mini_cluster.cc:1428] Setting key 3b51bc16794ca9d5b612375be04130ed
Aug 12 04:01:33 dist-test-slave-8tdl krb5kdc[20099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507293, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:33.242153 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.233691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:36529 (local address 127.31.43.254:40867)
0812 04:01:33.233749 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.233751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.233957 (+ 206us) server_negotiation.cc:407] Connection header received
0812 04:01:33.234749 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.234750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.234763 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.234823 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:33.235795 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.236288 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.237257 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.237442 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.237495 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.239806 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:33.239807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:33.239807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:33.239814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:33.240991 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:33.241357 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:33.241357 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:33.241358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:33.241407 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:33.241620 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:33.241620 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:33.241620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:33.241721 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:33.241773 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.242000 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.242084 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 04:01:33.242318 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.233658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:40867 (local address 127.31.43.195:36529)
0812 04:01:33.233834 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.233872 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.234596 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.234916 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.234920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.235506 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:33.235715 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.235719 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.236372 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.236373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.237178 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.237180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.237517 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.237518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.237632 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.238196 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:01:33.238205 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:33.239715 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:33.241074 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:33.241080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:33.241082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:33.241290 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:33.241465 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:33.241466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:33.241466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:33.241564 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:33.241776 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:33.241780 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:33.241891 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.241974 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.242112 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260812 04:01:33.242844 20651 heartbeater.cc:347] Connected to a master server at 127.31.43.254:40867
I20260812 04:01:33.242965 20651 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:33.243155 20651 heartbeater.cc:511] Master 127.31.43.254:40867 requested a full tablet report, sending...
I20260812 04:01:33.243544 20146 ts_manager.cc:194] Registered new tserver with Master: 2b8cf6fd792a443989b3d57a9abe6e8b (127.31.43.195:46609)
I20260812 04:01:33.243927 20146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:36529
I20260812 04:01:33.245527 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 04:01:33.253765 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.246915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.31.43.254:40867)
0812 04:01:33.246994 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.246997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.247294 (+ 297us) server_negotiation.cc:407] Connection header received
0812 04:01:33.247412 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.247413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.247424 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.247487 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:33.248064 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.248551 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.249422 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.249634 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.249710 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.251210 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:33.251211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:33.251212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:33.251219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:33.252410 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:33.252869 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:33.252871 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:33.252871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:33.252936 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:33.253171 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:33.253172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:33.253172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:33.253318 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:33.253436 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.253599 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.253663 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 04:01:33.255345 20146 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 04:01:33.256976 20146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 04:01:33.263383 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.259531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:46421 (local address 127.0.0.1:45152)
0812 04:01:33.259664 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.259673 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.259712 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.260026 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.260027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.260029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:33.260071 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.260073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.261064 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.261065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.262128 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.262130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.263080 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.263081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.263157 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.263171 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.263249 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.263299 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":99,"threads_started":1}
I20260812 04:01:33.263406 20666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.259683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45152 (local address 127.31.43.194:46421)
0812 04:01:33.259864 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.259869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.259879 (+ 10us) server_negotiation.cc:407] Connection header received
0812 04:01:33.259923 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.259925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.259942 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.260029 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:33.260126 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.260950 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.262222 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.262987 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.263058 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.263116 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.263275 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.263318 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260812 04:01:33.263701 20663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.259315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:33535 (local address 127.0.0.1:54298)
0812 04:01:33.259432 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.259472 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.259536 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.259904 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.259908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.259911 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:33.259986 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.259988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.261608 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.261609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.262625 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.262627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.263452 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.263453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.263523 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.263528 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.263578 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.263621 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260812 04:01:33.264040 20664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.259459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54298 (local address 127.31.43.193:33535)
0812 04:01:33.259692 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.259698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.259709 (+ 11us) server_negotiation.cc:407] Connection header received
0812 04:01:33.259756 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.259758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.259775 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.259892 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:33.260383 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.261497 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.262736 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.263354 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.263799 (+ 445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.263868 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.263896 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.263948 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":117,"threads_started":1}
I20260812 04:01:33.264122 20667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.259736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:46609 (local address 127.0.0.1:50600)
0812 04:01:33.259946 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.259956 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.260001 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.260550 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.260561 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.260563 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:33.260604 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.260606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.261821 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.261822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.262863 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.262866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.263889 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.263890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.263950 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.263954 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.264003 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.264039 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":74,"threads_started":1}
I20260812 04:01:33.264297 20427 tablet_service.cc:1467] Processing CreateTablet for tablet c12d5e1434b849dd8f8b24e913a5d14b (DEFAULT_TABLE table=test-table [id=72fa0f97dc6a4e5889973a3e3b559a48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:33.264635 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.260132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50600 (local address 127.31.43.195:46609)
0812 04:01:33.260319 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.260323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.260334 (+ 11us) server_negotiation.cc:407] Connection header received
0812 04:01:33.260374 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.260376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.260390 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.260471 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:33.260688 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.261720 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.262965 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.263725 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.263799 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.263852 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.264509 (+ 657us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.264553 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":119,"threads_started":1}
I20260812 04:01:33.264734 20427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c12d5e1434b849dd8f8b24e913a5d14b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:33.264797 20280 tablet_service.cc:1467] Processing CreateTablet for tablet c12d5e1434b849dd8f8b24e913a5d14b (DEFAULT_TABLE table=test-table [id=72fa0f97dc6a4e5889973a3e3b559a48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:33.265122 20280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c12d5e1434b849dd8f8b24e913a5d14b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:33.265232 20575 tablet_service.cc:1467] Processing CreateTablet for tablet c12d5e1434b849dd8f8b24e913a5d14b (DEFAULT_TABLE table=test-table [id=72fa0f97dc6a4e5889973a3e3b559a48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:33.265527 20575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c12d5e1434b849dd8f8b24e913a5d14b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:33.268792 20670 tablet_bootstrap.cc:492] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04: Bootstrap starting.
I20260812 04:01:33.268958 20669 tablet_bootstrap.cc:492] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9: Bootstrap starting.
I20260812 04:01:33.269527 20670 tablet_bootstrap.cc:654] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:33.269562 20669 tablet_bootstrap.cc:654] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:33.270033 20670 log.cc:824] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:33.270044 20669 log.cc:824] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:33.270419 20671 tablet_bootstrap.cc:492] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b: Bootstrap starting.
I20260812 04:01:33.270984 20670 tablet_bootstrap.cc:492] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04: No bootstrap required, opened a new log
I20260812 04:01:33.271078 20670 ts_tablet_manager.cc:1397] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 04:01:33.271176 20671 tablet_bootstrap.cc:654] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:33.271673 20669 tablet_bootstrap.cc:492] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9: No bootstrap required, opened a new log
I20260812 04:01:33.271757 20669 ts_tablet_manager.cc:1397] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 04:01:33.271749 20671 log.cc:824] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:33.272569 20671 tablet_bootstrap.cc:492] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b: No bootstrap required, opened a new log
I20260812 04:01:33.272651 20671 ts_tablet_manager.cc:1397] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 04:01:33.272931 20670 raft_consensus.cc:348] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 33535 } } peers { permanent_uuid: "c34c1f3c5fec40cd833b4eb41a73af04" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 46421 } } peers { permanent_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46609 } }
I20260812 04:01:33.273116 20670 raft_consensus.cc:374] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:33.273183 20670 raft_consensus.cc:729] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34c1f3c5fec40cd833b4eb41a73af04, State: Initialized, Role: FOLLOWER
I20260812 04:01:33.273315 20670 consensus_queue.cc:260] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 33535 } } peers { permanent_uuid: "c34c1f3c5fec40cd833b4eb41a73af04" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 46421 } } peers { permanent_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46609 } }
I20260812 04:01:33.273645 20501 heartbeater.cc:503] Master 127.31.43.254:40867 was elected leader, sending a full tablet report...
I20260812 04:01:33.273645 20670 ts_tablet_manager.cc:1428] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 04:01:33.273993 20669 raft_consensus.cc:348] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 33535 } } peers { permanent_uuid: "c34c1f3c5fec40cd833b4eb41a73af04" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 46421 } } peers { permanent_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46609 } }
I20260812 04:01:33.274174 20669 raft_consensus.cc:374] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:33.274211 20669 raft_consensus.cc:729] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfc6ebcddd4d430aaa6b166fee43bfa9, State: Initialized, Role: FOLLOWER
I20260812 04:01:33.274405 20669 consensus_queue.cc:260] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 33535 } } peers { permanent_uuid: "c34c1f3c5fec40cd833b4eb41a73af04" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 46421 } } peers { permanent_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46609 } }
I20260812 04:01:33.274798 20669 ts_tablet_manager.cc:1428] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 04:01:33.274879 20352 heartbeater.cc:503] Master 127.31.43.254:40867 was elected leader, sending a full tablet report...
I20260812 04:01:33.274760 20671 raft_consensus.cc:348] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 33535 } } peers { permanent_uuid: "c34c1f3c5fec40cd833b4eb41a73af04" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 46421 } } peers { permanent_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46609 } }
I20260812 04:01:33.274942 20671 raft_consensus.cc:374] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:33.274991 20671 raft_consensus.cc:729] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b8cf6fd792a443989b3d57a9abe6e8b, State: Initialized, Role: FOLLOWER
I20260812 04:01:33.275134 20671 consensus_queue.cc:260] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 33535 } } peers { permanent_uuid: "c34c1f3c5fec40cd833b4eb41a73af04" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 46421 } } peers { permanent_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46609 } }
I20260812 04:01:33.275426 20671 ts_tablet_manager.cc:1428] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 04:01:33.275508 20651 heartbeater.cc:503] Master 127.31.43.254:40867 was elected leader, sending a full tablet report...
I20260812 04:01:33.301503 20677 raft_consensus.cc:482] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:01:33.301644 20677 raft_consensus.cc:504] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 33535 } } peers { permanent_uuid: "c34c1f3c5fec40cd833b4eb41a73af04" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 46421 } } peers { permanent_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46609 } }
I20260812 04:01:33.302098 20677 leader_election.cc:288] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfc6ebcddd4d430aaa6b166fee43bfa9 (127.31.43.193:33535), c34c1f3c5fec40cd833b4eb41a73af04 (127.31.43.194:46421)
I20260812 04:01:33.305596 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.302264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:33535 (local address 127.31.43.195:35191)
0812 04:01:33.302344 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.302356 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.302433 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.302654 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.302655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.302657 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:33.302762 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.302763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.303728 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.303729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.304554 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.304556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.305277 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.305278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.305359 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.305381 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.305456 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.305502 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 04:01:33.305611 20664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.302369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:35191 (local address 127.31.43.193:33535)
0812 04:01:33.302468 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.302471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.302483 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:01:33.302515 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.302516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.302527 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.302588 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:33.302835 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.303626 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.304676 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.305157 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.305232 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.305244 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.305476 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.305528 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 04:01:33.305855 20678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.302384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:46421 (local address 127.31.43.195:47671)
0812 04:01:33.302615 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.302623 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.302720 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.302877 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.302878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.302879 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:33.302926 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.302927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.303763 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.303764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.304761 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.304764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.305589 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.305590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.305673 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.305678 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.305758 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.305792 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":140,"threads_started":1}
I20260812 04:01:33.305879 20666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.302504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:47671 (local address 127.31.43.194:46421)
0812 04:01:33.302555 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.302557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.302684 (+ 127us) server_negotiation.cc:407] Connection header received
0812 04:01:33.302806 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.302807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.302818 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.302872 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:33.302989 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.303694 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.304861 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.305375 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.305473 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.305482 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.305765 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.305814 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 04:01:33.306242 20303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c12d5e1434b849dd8f8b24e913a5d14b" candidate_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9" is_pre_election: true
I20260812 04:01:33.306239 20450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c12d5e1434b849dd8f8b24e913a5d14b" candidate_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34c1f3c5fec40cd833b4eb41a73af04" is_pre_election: true
I20260812 04:01:33.306375 20303 raft_consensus.cc:2454] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b8cf6fd792a443989b3d57a9abe6e8b in term 0.
I20260812 04:01:33.306408 20450 raft_consensus.cc:2454] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b8cf6fd792a443989b3d57a9abe6e8b in term 0.
I20260812 04:01:33.306721 20528 leader_election.cc:302] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b8cf6fd792a443989b3d57a9abe6e8b, c34c1f3c5fec40cd833b4eb41a73af04; no voters:
I20260812 04:01:33.306859 20677 raft_consensus.cc:2781] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 04:01:33.306918 20677 raft_consensus.cc:482] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 04:01:33.306952 20677 raft_consensus.cc:3037] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:33.307653 20677 raft_consensus.cc:504] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 33535 } } peers { permanent_uuid: "c34c1f3c5fec40cd833b4eb41a73af04" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 46421 } } peers { permanent_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46609 } }
I20260812 04:01:33.307790 20677 leader_election.cc:288] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [CANDIDATE]: Term 1 election: Requested vote from peers cfc6ebcddd4d430aaa6b166fee43bfa9 (127.31.43.193:33535), c34c1f3c5fec40cd833b4eb41a73af04 (127.31.43.194:46421)
I20260812 04:01:33.307950 20303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c12d5e1434b849dd8f8b24e913a5d14b" candidate_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9"
I20260812 04:01:33.308027 20450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c12d5e1434b849dd8f8b24e913a5d14b" candidate_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34c1f3c5fec40cd833b4eb41a73af04"
I20260812 04:01:33.308045 20303 raft_consensus.cc:3037] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:33.308123 20450 raft_consensus.cc:3037] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:33.309109 20450 raft_consensus.cc:2454] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b8cf6fd792a443989b3d57a9abe6e8b in term 1.
I20260812 04:01:33.309109 20303 raft_consensus.cc:2454] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b8cf6fd792a443989b3d57a9abe6e8b in term 1.
I20260812 04:01:33.309321 20528 leader_election.cc:302] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b8cf6fd792a443989b3d57a9abe6e8b, c34c1f3c5fec40cd833b4eb41a73af04; no voters:
I20260812 04:01:33.309424 20677 raft_consensus.cc:2781] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:33.309597 20677 raft_consensus.cc:686] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 1 LEADER]: Becoming Leader. State: Replica: 2b8cf6fd792a443989b3d57a9abe6e8b, State: Running, Role: LEADER
I20260812 04:01:33.309676 20677 consensus_queue.cc:237] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 33535 } } peers { permanent_uuid: "c34c1f3c5fec40cd833b4eb41a73af04" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 46421 } } peers { permanent_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46609 } }
I20260812 04:01:33.310623 20145 catalog_manager.cc:5495] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b8cf6fd792a443989b3d57a9abe6e8b (127.31.43.195). New cstate: current_term: 1 leader_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 33535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c34c1f3c5fec40cd833b4eb41a73af04" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 46421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b8cf6fd792a443989b3d57a9abe6e8b" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46609 } health_report { overall_health: HEALTHY } } }
W20260812 04:01:33.317140 20502 tablet.cc:2369] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:01:33.322228 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.319829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50610 (local address 127.31.43.195:46609)
0812 04:01:33.319890 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.319892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.319901 (+ 9us) server_negotiation.cc:407] Connection header received
0812 04:01:33.319982 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.319982 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.319999 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.320083 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:01:33.320240 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.320684 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.321424 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.321615 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.321673 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.321856 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:01:33.321937 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:01:33.322006 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.322072 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.322107 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 04:01:33.324136 20450 raft_consensus.cc:1261] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04 [term 1 FOLLOWER]: Refusing update from remote peer 2b8cf6fd792a443989b3d57a9abe6e8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:01:33.324177 20303 raft_consensus.cc:1261] T c12d5e1434b849dd8f8b24e913a5d14b P cfc6ebcddd4d430aaa6b166fee43bfa9 [term 1 FOLLOWER]: Refusing update from remote peer 2b8cf6fd792a443989b3d57a9abe6e8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:01:33.324586 20677 consensus_queue.cc:1035] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c34c1f3c5fec40cd833b4eb41a73af04" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 46421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:01:33.324775 20679 consensus_queue.cc:1035] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfc6ebcddd4d430aaa6b166fee43bfa9" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 33535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:01:33.327142 20684 mvcc.cc:205] Tried to move back new op lower bound from 7317533873452990464 to 7317533873397288960. Current Snapshot: MvccSnapshot[applied={T|T < 7317533873452990464}]
I20260812 04:01:33.327260 20685 mvcc.cc:205] Tried to move back new op lower bound from 7317533873452990464 to 7317533873397288960. Current Snapshot: MvccSnapshot[applied={T|T < 7317533873452990464}]
I20260812 04:01:33.327761 20686 mvcc.cc:205] Tried to move back new op lower bound from 7317533873452990464 to 7317533873397288960. Current Snapshot: MvccSnapshot[applied={T|T < 7317533873452990464}]
I20260812 04:01:33.329542 20146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:01:33.329633 20146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:01:33.330375 20146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd3c838368594413a2defeeca4a8ac5d: Sending DeleteTablet for 3 replicas of tablet c12d5e1434b849dd8f8b24e913a5d14b
I20260812 04:01:33.330726 20280 tablet_service.cc:1514] Processing DeleteTablet for tablet c12d5e1434b849dd8f8b24e913a5d14b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:33 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:54298
I20260812 04:01:33.330775 20575 tablet_service.cc:1514] Processing DeleteTablet for tablet c12d5e1434b849dd8f8b24e913a5d14b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:33 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:50600
I20260812 04:01:33.330785 20427 tablet_service.cc:1514] Processing DeleteTablet for tablet c12d5e1434b849dd8f8b24e913a5d14b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:33 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:45152
I20260812 04:01:33.331089 20699 tablet_replica.cc:330] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04: stopping tablet replica
I20260812 04:01:33.331125 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 20215
I20260812 04:01:33.331205 20699 raft_consensus.cc:2229] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:01:33.331746 20699 raft_consensus.cc:2258] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:33.332950 20699 ts_tablet_manager.cc:1905] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:01:33.335345 20699 ts_tablet_manager.cc:1918] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:01:33.335418 20699 log.cc:1196] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/c12d5e1434b849dd8f8b24e913a5d14b
I20260812 04:01:33.335697 20699 ts_tablet_manager.cc:1939] T c12d5e1434b849dd8f8b24e913a5d14b P c34c1f3c5fec40cd833b4eb41a73af04: Deleting consensus metadata
I20260812 04:01:33.336238 20132 catalog_manager.cc:4841] TS c34c1f3c5fec40cd833b4eb41a73af04 (127.31.43.194:46421): tablet c12d5e1434b849dd8f8b24e913a5d14b (table test-table [id=72fa0f97dc6a4e5889973a3e3b559a48]) successfully deleted
I20260812 04:01:33.342839 20698 tablet_replica.cc:330] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b: stopping tablet replica
I20260812 04:01:33.342993 20698 raft_consensus.cc:2229] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 1 LEADER]: Raft consensus shutting down.
I20260812 04:01:33.343155 20698 raft_consensus.cc:2258] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:33.343613 20698 ts_tablet_manager.cc:1905] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 04:01:33.345376 20132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:33535 (error 108)
I20260812 04:01:33.345583 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 20357
I20260812 04:01:33.345759 20698 ts_tablet_manager.cc:1918] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:01:33.345829 20698 log.cc:1196] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/c12d5e1434b849dd8f8b24e913a5d14b
I20260812 04:01:33.346004 20667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.345858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:33535 (local address 127.0.0.1:54312)
0812 04:01:33.345915 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.345958 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:33535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 04:01:33.346125 20132 catalog_manager.cc:4584] TS cfc6ebcddd4d430aaa6b166fee43bfa9 (127.31.43.193:33535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c12d5e1434b849dd8f8b24e913a5d14b: Network error: Client connection negotiation failed: client connection to 127.31.43.193:33535: connect: Connection refused (error 111)
I20260812 04:01:33.346856 20698 ts_tablet_manager.cc:1939] T c12d5e1434b849dd8f8b24e913a5d14b P 2b8cf6fd792a443989b3d57a9abe6e8b: Deleting consensus metadata
I20260812 04:01:33.347713 20132 catalog_manager.cc:4841] TS 2b8cf6fd792a443989b3d57a9abe6e8b (127.31.43.195:46609): tablet c12d5e1434b849dd8f8b24e913a5d14b (table test-table [id=72fa0f97dc6a4e5889973a3e3b559a48]) successfully deleted
I20260812 04:01:33.352291 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 20506
I20260812 04:01:33.359369 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 20113
2026-08-12T04:01:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2129 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T04:01:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:33Z Disabled control of system clock
I20260812 04:01:33.391063 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:34507
--webserver_interface=127.31.43.254
--webserver_port=0
--builtin_ntp_servers=127.31.43.212:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:34507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 04:01:33.477387 20704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:33.477615 20704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:33.477638 20704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:33.478771 20704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:33.478811 20704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:33.478824 20704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:33.478837 20704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:33.478849 20704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:33.480160 20704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:34507
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:34507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:33.480369 20704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:33.480625 20704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 04:01:33.485811 20704 server_base.cc:1034] running on GCE node
W20260812 04:01:33.485733 20710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:33.485726 20709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:33.485785 20712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:33.486311 20704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:33.486588 20704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:33.487803 20704 hybrid_clock.cc:648] HybridClock initialized: now 1786507293487781 us; error 39 us; skew 500 ppm
I20260812 04:01:33.489092 20704 webserver.cc:460] Webserver started at http://127.31.43.254:43831/ using document root <none> and password file <none>
I20260812 04:01:33.489296 20704 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:33.489339 20704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:33.489429 20704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:33.490478 20704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "87b5a2a1be8e42a88527164f4058283f"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "2d3a0f1e6523bda47cdc4ae611ddfb39"
server_key_iv: "ca1ee02825f22f78c36e6b7bfad24cef"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.490869 20704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "87b5a2a1be8e42a88527164f4058283f"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "2d3a0f1e6523bda47cdc4ae611ddfb39"
server_key_iv: "ca1ee02825f22f78c36e6b7bfad24cef"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.492264 20704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:33.493007 20719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.493211 20704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:33.493302 20704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "87b5a2a1be8e42a88527164f4058283f"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "2d3a0f1e6523bda47cdc4ae611ddfb39"
server_key_iv: "ca1ee02825f22f78c36e6b7bfad24cef"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.493362 20704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:33.502106 20704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:33.502378 20704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:33.502485 20704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:33.506389 20704 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:34507
I20260812 04:01:33.506456 20775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:34507 every 8 connection(s)
I20260812 04:01:33.506825 20704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:33.507887 20776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:33.510231 20776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f: Bootstrap starting.
I20260812 04:01:33.510915 20776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:33.511226 20776 log.cc:824] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:33.511919 20776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f: No bootstrap required, opened a new log
I20260812 04:01:33.513543 20776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b5a2a1be8e42a88527164f4058283f" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34507 } }
I20260812 04:01:33.513700 20776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:33.513724 20776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87b5a2a1be8e42a88527164f4058283f, State: Initialized, Role: FOLLOWER
I20260812 04:01:33.513878 20776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b5a2a1be8e42a88527164f4058283f" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34507 } }
I20260812 04:01:33.513955 20776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:33.513983 20776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:33.514056 20776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:33.514611 20776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b5a2a1be8e42a88527164f4058283f" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34507 } }
I20260812 04:01:33.514786 20776 leader_election.cc:302] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87b5a2a1be8e42a88527164f4058283f; no voters:
I20260812 04:01:33.515008 20776 leader_election.cc:288] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:33.515074 20779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:33.515282 20776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:33.515311 20779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [term 1 LEADER]: Becoming Leader. State: Replica: 87b5a2a1be8e42a88527164f4058283f, State: Running, Role: LEADER
I20260812 04:01:33.515477 20779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b5a2a1be8e42a88527164f4058283f" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34507 } }
I20260812 04:01:33.515715 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 20704
I20260812 04:01:33.515841 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:33.516100 31919 external_mini_cluster.cc:1428] Setting key 071025344f09978e56f660cc3bf7d113
I20260812 04:01:33.516134 20782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87b5a2a1be8e42a88527164f4058283f. Latest consensus state: current_term: 1 leader_uuid: "87b5a2a1be8e42a88527164f4058283f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b5a2a1be8e42a88527164f4058283f" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34507 } } }
I20260812 04:01:33.516228 20782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:33.516463 20781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87b5a2a1be8e42a88527164f4058283f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b5a2a1be8e42a88527164f4058283f" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 34507 } } }
I20260812 04:01:33.516568 20781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f [sys.catalog]: This master's current role is: LEADER
W20260812 04:01:33.517427 20799 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:01:33.517513 20799 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:01:33.517590 20790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:33.517774 20790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:33.519507 20790 catalog_manager.cc:1347] Generated new cluster ID: 8bba88256da744d4bd4d9b0ac29409de
I20260812 04:01:33.519591 20790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:33.522712 20800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.516828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.31.43.254:34507)
0812 04:01:33.517020 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.517029 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.517076 (+ 47us) server_negotiation.cc:407] Connection header received
0812 04:01:33.517207 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.517215 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.517288 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.517398 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:33.517673 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.519214 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.520352 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.521025 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.521154 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.521964 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:33.521968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:33.521969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:33.522179 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:33.522221 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:33.522235 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:33.522306 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.522394 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.522472 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260812 04:01:33.526175 20790 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:33.526913 20790 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:33.539369 20790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f: Generated new TSK 0
I20260812 04:01:33.539592 20790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:33.544740 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:34507
--builtin_ntp_servers=127.31.43.212:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 04:01:33.633754 20803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:33.634025 20803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:33.634052 20803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:33.635319 20803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:33.635361 20803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:33.635387 20803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:01:33.636797 20803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:34507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:33.637053 20803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:33.637424 20803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:33.643764 20811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:33.643782 20809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:33.643980 20803 server_base.cc:1034] running on GCE node
W20260812 04:01:33.643764 20808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:33.644294 20803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:33.644564 20803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:33.645738 20803 hybrid_clock.cc:648] HybridClock initialized: now 1786507293645712 us; error 47 us; skew 500 ppm
I20260812 04:01:33.647107 20803 webserver.cc:460] Webserver started at http://127.31.43.193:39971/ using document root <none> and password file <none>
I20260812 04:01:33.647368 20803 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:33.647442 20803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:33.647604 20803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:33.648706 20803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "0e98f4a399dd40da8db0c44bac9424fd"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "2f8625c8ad18c80f21151c1f6e02071a"
server_key_iv: "d909e66940c96a2a3e3c43cc97b55d1b"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.649098 20803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "0e98f4a399dd40da8db0c44bac9424fd"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "2f8625c8ad18c80f21151c1f6e02071a"
server_key_iv: "d909e66940c96a2a3e3c43cc97b55d1b"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.650480 20803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:33.651381 20817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.651557 20803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:33.651667 20803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "0e98f4a399dd40da8db0c44bac9424fd"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "2f8625c8ad18c80f21151c1f6e02071a"
server_key_iv: "d909e66940c96a2a3e3c43cc97b55d1b"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.651753 20803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:33.659142 20803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:33.659469 20803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:33.659628 20803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:33.659857 20803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:33.660208 20803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:33.660265 20803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.660307 20803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:33.660336 20803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.666404 20803 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:45541
I20260812 04:01:33.666489 20942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:45541 every 8 connection(s)
I20260812 04:01:33.666900 20803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 04:01:33.672242 20800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.667767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:45427 (local address 127.31.43.254:34507)
0812 04:01:33.667876 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.667878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.668108 (+ 230us) server_negotiation.cc:407] Connection header received
0812 04:01:33.668598 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.668599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.668611 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.668675 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:33.669376 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.669872 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.670891 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.671096 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.671156 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.671982 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:33.671984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:33.671984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:33.671989 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:33.672005 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:33.672009 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:33.672071 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.672151 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.672179 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 04:01:33.672441 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.667733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:34507 (local address 127.31.43.193:45427)
0812 04:01:33.667962 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.668007 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.668513 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.668741 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.668745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.668752 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:33.669295 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.669300 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.669955 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.669956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.670806 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.670809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.671183 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.671184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.671306 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.671848 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 04:01:33.671848 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:33.671877 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 04:01:33.671900 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:33.672081 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:33.672086 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:33.672089 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.672139 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.672224 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":125,"threads_started":1}
I20260812 04:01:33.672986 20943 heartbeater.cc:347] Connected to a master server at 127.31.43.254:34507
I20260812 04:01:33.673091 20943 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:33.673246 20943 heartbeater.cc:511] Master 127.31.43.254:34507 requested a full tablet report, sending...
I20260812 04:01:33.673900 20736 ts_manager.cc:194] Registered new tserver with Master: 0e98f4a399dd40da8db0c44bac9424fd (127.31.43.193:45541)
I20260812 04:01:33.674175 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 20803
I20260812 04:01:33.674292 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 04:01:33.674492 31919 external_mini_cluster.cc:1428] Setting key 05ac0fe28732e2250b3f363544282d30
I20260812 04:01:33.674800 20736 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.43.193:45427
I20260812 04:01:33.675746 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:34507
--builtin_ntp_servers=127.31.43.212:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 04:01:33.765393 20947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:33.765633 20947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:33.765657 20947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:33.766877 20947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:33.766919 20947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:33.766947 20947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:01:33.768360 20947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:34507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:33.768628 20947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:33.768945 20947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:33.774371 20952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:33.774313 20955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:33.774348 20953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:33.774493 20947 server_base.cc:1034] running on GCE node
I20260812 04:01:33.774855 20947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:33.775094 20947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:33.776247 20947 hybrid_clock.cc:648] HybridClock initialized: now 1786507293776221 us; error 36 us; skew 500 ppm
I20260812 04:01:33.777635 20947 webserver.cc:460] Webserver started at http://127.31.43.194:46469/ using document root <none> and password file <none>
I20260812 04:01:33.777916 20947 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:33.777993 20947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:33.778179 20947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:33.779340 20947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "aee481b2b30f4276bd84302ec8e06e41"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "baef6727e213df47393522e3ee74c6b7"
server_key_iv: "fb022016de6d524a566555d8aec86ea3"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.779707 20947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "aee481b2b30f4276bd84302ec8e06e41"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "baef6727e213df47393522e3ee74c6b7"
server_key_iv: "fb022016de6d524a566555d8aec86ea3"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.781033 20947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:01:33.781809 20962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.781985 20947 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 04:01:33.782104 20947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "aee481b2b30f4276bd84302ec8e06e41"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "baef6727e213df47393522e3ee74c6b7"
server_key_iv: "fb022016de6d524a566555d8aec86ea3"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.782181 20947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:33.795854 20947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:33.796221 20947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:33.796418 20947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:33.796728 20947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:33.797125 20947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:33.797183 20947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.797226 20947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:33.797252 20947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.803532 20947 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:44139
I20260812 04:01:33.803593 21088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:44139 every 8 connection(s)
I20260812 04:01:33.803972 20947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:01:33.809613 20800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.804917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:58441 (local address 127.31.43.254:34507)
0812 04:01:33.805013 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.805015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.805201 (+ 186us) server_negotiation.cc:407] Connection header received
0812 04:01:33.805758 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.805759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.805772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.805833 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:33.806540 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.807143 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.808136 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.808346 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.808427 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.809312 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:33.809314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:33.809314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:33.809319 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:33.809335 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:33.809339 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:33.809394 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.809499 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.809530 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 04:01:33.809859 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.804898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:34507 (local address 127.31.43.194:58441)
0812 04:01:33.805078 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.805121 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.805645 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.805906 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.805910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.805919 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:33.806441 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.806445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.807242 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.807243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.808041 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.808043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.808437 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.808438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.808572 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.809162 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 04:01:33.809162 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:33.809192 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 04:01:33.809215 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:33.809404 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:33.809409 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:33.809412 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.809499 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.809602 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260812 04:01:33.810324 21089 heartbeater.cc:347] Connected to a master server at 127.31.43.254:34507
I20260812 04:01:33.810420 21089 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:33.810621 21089 heartbeater.cc:511] Master 127.31.43.254:34507 requested a full tablet report, sending...
I20260812 04:01:33.811122 20735 ts_manager.cc:194] Registered new tserver with Master: aee481b2b30f4276bd84302ec8e06e41 (127.31.43.194:44139)
I20260812 04:01:33.811548 20735 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.43.194:58441
I20260812 04:01:33.811798 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 20947
I20260812 04:01:33.811909 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 04:01:33.812175 31919 external_mini_cluster.cc:1428] Setting key 90c54d0dc839f56d131f08c9c45eec9d
I20260812 04:01:33.813174 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:34507
--builtin_ntp_servers=127.31.43.212:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 04:01:33.900852 21092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:33.901095 21092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:33.901134 21092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:33.902281 21092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:33.902345 21092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:33.902385 21092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:01:33.903774 21092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:34507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:33.904047 21092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:33.904351 21092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:33.909566 21098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:33.909592 21100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:33.909619 21092 server_base.cc:1034] running on GCE node
W20260812 04:01:33.909592 21097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:33.910084 21092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:33.910348 21092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:33.911572 21092 hybrid_clock.cc:648] HybridClock initialized: now 1786507293911536 us; error 39 us; skew 500 ppm
I20260812 04:01:33.912887 21092 webserver.cc:460] Webserver started at http://127.31.43.195:43165/ using document root <none> and password file <none>
I20260812 04:01:33.913149 21092 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:33.913224 21092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:33.913380 21092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:33.914474 21092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "4cf3c2c39f484e81956e2b5a6c473b15"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "9ad6afbcc299c7cb803c875ddce41add"
server_key_iv: "f31d49b29d16c91b83c83e93f69086e5"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.914904 21092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "4cf3c2c39f484e81956e2b5a6c473b15"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "9ad6afbcc299c7cb803c875ddce41add"
server_key_iv: "f31d49b29d16c91b83c83e93f69086e5"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.916440 21092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:33.917244 21106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.917430 21092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:33.917570 21092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "4cf3c2c39f484e81956e2b5a6c473b15"
format_stamp: "Formatted at 2026-08-12 04:01:33 on dist-test-slave-8tdl"
server_key: "9ad6afbcc299c7cb803c875ddce41add"
server_key_iv: "f31d49b29d16c91b83c83e93f69086e5"
server_key_version: "encryptionkey@0"
I20260812 04:01:33.917650 21092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:33.937001 21092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:33.937346 21092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:33.937510 21092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:33.937777 21092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:33.938133 21092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:33.938189 21092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.938226 21092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:33.938273 21092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:33.944180 21092 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:46651
I20260812 04:01:33.944293 21230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:46651 every 8 connection(s)
I20260812 04:01:33.944629 21092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 04:01:33.949549 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 21092
I20260812 04:01:33.949677 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 04:01:33.949920 31919 external_mini_cluster.cc:1428] Setting key b0fc8596e8b3ede1aa16ad77f6ce30f7
I20260812 04:01:33.950287 20800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.945479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:57635 (local address 127.31.43.254:34507)
0812 04:01:33.945558 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.945562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.945807 (+ 245us) server_negotiation.cc:407] Connection header received
0812 04:01:33.946316 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.946317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.946329 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.946392 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:33.947197 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.947704 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.948675 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.948871 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.948930 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.949870 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:33.949872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:33.949872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:33.949877 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:33.949946 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:33.949950 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:33.950029 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.950155 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.950185 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 04:01:33.950608 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.945467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:34507 (local address 127.31.43.195:57635)
0812 04:01:33.945666 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.945709 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.946214 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.946446 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.946449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.946457 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:33.947097 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.947103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.947790 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.947791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.948585 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.948587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.948953 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.948954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.949067 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.949696 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 04:01:33.949697 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:33.949738 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 04:01:33.949768 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:33.950058 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:33.950065 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:33.950069 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.950152 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.950273 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260812 04:01:33.951087 21231 heartbeater.cc:347] Connected to a master server at 127.31.43.254:34507
I20260812 04:01:33.951195 21231 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:33.951392 21231 heartbeater.cc:511] Master 127.31.43.254:34507 requested a full tablet report, sending...
I20260812 04:01:33.951841 20735 ts_manager.cc:194] Registered new tserver with Master: 4cf3c2c39f484e81956e2b5a6c473b15 (127.31.43.195:46651)
I20260812 04:01:33.952234 20735 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.43.195:57635
I20260812 04:01:33.962126 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 04:01:33.966473 20800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.963714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35626 (local address 127.31.43.254:34507)
0812 04:01:33.963790 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.963791 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.963911 (+ 120us) server_negotiation.cc:407] Connection header received
0812 04:01:33.963948 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.963949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.963960 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.964009 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:33.964159 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.964638 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.965367 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.965546 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.965596 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.966234 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:33.966236 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:33.966236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:33.966240 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:33.966255 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:33.966259 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:33.966338 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.966403 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.966414 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 04:01:33.968209 20735 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 04:01:33.969753 20735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 04:01:33.976735 21243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.971991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:46651 (local address 127.0.0.1:59742)
0812 04:01:33.972219 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.972259 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.972327 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.972706 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.972709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.972711 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:33.972781 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.972783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.974110 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.974111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.975466 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.975470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.976395 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.976396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.976493 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.976510 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.976594 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.976653 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":11,"thread_start_us":114,"threads_started":1}
I20260812 04:01:33.976750 21245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.972328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.31.43.195:46651)
0812 04:01:33.972509 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.972514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.972525 (+ 11us) server_negotiation.cc:407] Connection header received
0812 04:01:33.972579 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.972581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.972604 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.972698 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:33.972870 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.974004 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.975570 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.976301 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.976383 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.976480 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.976623 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.976664 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260812 04:01:33.977718 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.972002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45541 (local address 127.0.0.1:35090)
0812 04:01:33.972435 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.972445 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.972485 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.972985 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.972986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.972990 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:33.973055 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.973058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.975325 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.975326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.976395 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.976398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.977460 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.977461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.977522 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.977527 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.977599 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.977645 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":21,"thread_start_us":112,"threads_started":1}
I20260812 04:01:33.977607 21158 tablet_service.cc:1467] Processing CreateTablet for tablet 33d78d8e31b4417ea7e95a5767d8a0f2 (DEFAULT_TABLE table=test-table [id=4c5d61d7175049fb88bb12825314bf47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:33.977782 21244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.972300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35090 (local address 127.31.43.193:45541)
0812 04:01:33.972751 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.972755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.972765 (+ 10us) server_negotiation.cc:407] Connection header received
0812 04:01:33.972806 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.972808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.972824 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.972912 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:33.974295 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.975230 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.976485 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.977271 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.977341 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.977422 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.977644 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.977695 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":93,"threads_started":1}
I20260812 04:01:33.978067 21158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33d78d8e31b4417ea7e95a5767d8a0f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:33.978374 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.971990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:44139 (local address 127.0.0.1:34738)
0812 04:01:33.972219 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:33.972259 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:33.972327 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:33.972986 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:33.972986 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:33.972989 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:33.973055 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.973058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.974305 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.974306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.975466 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:33.975470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.978069 (+ 2599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.978070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:33.978204 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.978209 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 04:01:33.978261 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 04:01:33.978303 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":6,"thread_start_us":116,"threads_started":1}
I20260812 04:01:33.978664 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:33.972514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34738 (local address 127.31.43.194:44139)
0812 04:01:33.972692 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:33.972697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:33.972709 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:01:33.972816 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:33.972819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:33.972832 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:33.972912 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:33.973133 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.974206 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.976529 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:33.977618 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:33.977693 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:33.978493 (+ 800us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:33.978523 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 04:01:33.978582 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
I20260812 04:01:33.978551 20868 tablet_service.cc:1467] Processing CreateTablet for tablet 33d78d8e31b4417ea7e95a5767d8a0f2 (DEFAULT_TABLE table=test-table [id=4c5d61d7175049fb88bb12825314bf47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:33.978947 20868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33d78d8e31b4417ea7e95a5767d8a0f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:33.981151 21247 tablet_bootstrap.cc:492] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15: Bootstrap starting.
I20260812 04:01:33.981755 21249 tablet_bootstrap.cc:492] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd: Bootstrap starting.
I20260812 04:01:33.981747 21013 tablet_service.cc:1467] Processing CreateTablet for tablet 33d78d8e31b4417ea7e95a5767d8a0f2 (DEFAULT_TABLE table=test-table [id=4c5d61d7175049fb88bb12825314bf47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:33.982088 21247 tablet_bootstrap.cc:654] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:33.982108 21013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33d78d8e31b4417ea7e95a5767d8a0f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:33.982417 21249 tablet_bootstrap.cc:654] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:33.982448 21247 log.cc:824] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:33.982826 21249 log.cc:824] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:33.983317 21247 tablet_bootstrap.cc:492] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15: No bootstrap required, opened a new log
I20260812 04:01:33.983398 21247 ts_tablet_manager.cc:1397] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 04:01:33.983614 21249 tablet_bootstrap.cc:492] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd: No bootstrap required, opened a new log
I20260812 04:01:33.983696 21249 ts_tablet_manager.cc:1397] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 04:01:33.985661 21253 tablet_bootstrap.cc:492] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41: Bootstrap starting.
I20260812 04:01:33.985965 21249 raft_consensus.cc:348] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee481b2b30f4276bd84302ec8e06e41" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 44139 } } peers { permanent_uuid: "4cf3c2c39f484e81956e2b5a6c473b15" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46651 } } peers { permanent_uuid: "0e98f4a399dd40da8db0c44bac9424fd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45541 } }
I20260812 04:01:33.986145 21249 raft_consensus.cc:374] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:33.986191 21249 raft_consensus.cc:729] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e98f4a399dd40da8db0c44bac9424fd, State: Initialized, Role: FOLLOWER
I20260812 04:01:33.986078 21247 raft_consensus.cc:348] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee481b2b30f4276bd84302ec8e06e41" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 44139 } } peers { permanent_uuid: "4cf3c2c39f484e81956e2b5a6c473b15" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46651 } } peers { permanent_uuid: "0e98f4a399dd40da8db0c44bac9424fd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45541 } }
I20260812 04:01:33.986239 21247 raft_consensus.cc:374] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:33.986290 21247 raft_consensus.cc:729] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cf3c2c39f484e81956e2b5a6c473b15, State: Initialized, Role: FOLLOWER
I20260812 04:01:33.986313 21253 tablet_bootstrap.cc:654] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:33.986337 21249 consensus_queue.cc:260] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee481b2b30f4276bd84302ec8e06e41" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 44139 } } peers { permanent_uuid: "4cf3c2c39f484e81956e2b5a6c473b15" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46651 } } peers { permanent_uuid: "0e98f4a399dd40da8db0c44bac9424fd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45541 } }
I20260812 04:01:33.986421 21247 consensus_queue.cc:260] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee481b2b30f4276bd84302ec8e06e41" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 44139 } } peers { permanent_uuid: "4cf3c2c39f484e81956e2b5a6c473b15" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46651 } } peers { permanent_uuid: "0e98f4a399dd40da8db0c44bac9424fd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45541 } }
I20260812 04:01:33.986634 21253 log.cc:824] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:33.986661 21249 ts_tablet_manager.cc:1428] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 04:01:33.986739 21247 ts_tablet_manager.cc:1428] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260812 04:01:33.986819 20943 heartbeater.cc:503] Master 127.31.43.254:34507 was elected leader, sending a full tablet report...
I20260812 04:01:33.986860 21231 heartbeater.cc:503] Master 127.31.43.254:34507 was elected leader, sending a full tablet report...
I20260812 04:01:33.987733 21253 tablet_bootstrap.cc:492] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41: No bootstrap required, opened a new log
I20260812 04:01:33.987844 21253 ts_tablet_manager.cc:1397] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 04:01:33.989845 21253 raft_consensus.cc:348] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee481b2b30f4276bd84302ec8e06e41" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 44139 } } peers { permanent_uuid: "4cf3c2c39f484e81956e2b5a6c473b15" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46651 } } peers { permanent_uuid: "0e98f4a399dd40da8db0c44bac9424fd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45541 } }
I20260812 04:01:33.990024 21253 raft_consensus.cc:374] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:33.990061 21253 raft_consensus.cc:729] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aee481b2b30f4276bd84302ec8e06e41, State: Initialized, Role: FOLLOWER
I20260812 04:01:33.990149 21253 consensus_queue.cc:260] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee481b2b30f4276bd84302ec8e06e41" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 44139 } } peers { permanent_uuid: "4cf3c2c39f484e81956e2b5a6c473b15" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46651 } } peers { permanent_uuid: "0e98f4a399dd40da8db0c44bac9424fd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45541 } }
I20260812 04:01:33.990343 21253 ts_tablet_manager.cc:1428] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 04:01:33.990433 21089 heartbeater.cc:503] Master 127.31.43.254:34507 was elected leader, sending a full tablet report...
I20260812 04:01:34.042531 21257 raft_consensus.cc:482] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:01:34.042727 21257 raft_consensus.cc:504] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee481b2b30f4276bd84302ec8e06e41" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 44139 } } peers { permanent_uuid: "4cf3c2c39f484e81956e2b5a6c473b15" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46651 } } peers { permanent_uuid: "0e98f4a399dd40da8db0c44bac9424fd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45541 } }
I20260812 04:01:34.043161 21257 leader_election.cc:288] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cf3c2c39f484e81956e2b5a6c473b15 (127.31.43.195:46651), 0e98f4a399dd40da8db0c44bac9424fd (127.31.43.193:45541)
I20260812 04:01:34.046277 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:34.043360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45541 (local address 127.31.43.194:54693)
0812 04:01:34.043423 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:34.043435 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:34.043482 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:34.043664 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:34.043665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:34.043667 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:34.043750 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:34.043752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:34.044602 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:34.044603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:34.045456 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:34.045459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:34.046010 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:34.046011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:34.046087 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:34.046091 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:01:34.046147 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 04:01:34.046182 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":49}
I20260812 04:01:34.046330 21244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:34.043444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:54693 (local address 127.31.43.193:45541)
0812 04:01:34.043492 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:34.043494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:34.043507 (+ 13us) server_negotiation.cc:407] Connection header received
0812 04:01:34.043552 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:34.043553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:34.043566 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:34.043625 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:34.043839 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:34.044401 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:34.045548 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:34.045930 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:34.045988 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:34.045994 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:34.046179 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 04:01:34.046229 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 04:01:34.046397 21245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:34.043450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:48337 (local address 127.31.43.195:46651)
0812 04:01:34.043511 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:34.043513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:34.043668 (+ 155us) server_negotiation.cc:407] Connection header received
0812 04:01:34.043704 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:34.043705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:34.043715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:34.043787 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:34.043989 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:34.044591 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:34.045621 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:34.045994 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:34.046079 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:34.046086 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:34.046268 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 04:01:34.046320 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 04:01:34.046424 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:34.043367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:46651 (local address 127.31.43.194:48337)
0812 04:01:34.043591 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:34.043600 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:34.043652 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:34.043827 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:34.043828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:34.043830 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:34.043903 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:34.043904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:34.044674 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:34.044675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:34.045488 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:34.045514 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:34.046089 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:34.046090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:34.046182 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:34.046187 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 04:01:34.046242 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 04:01:34.046277 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":45,"thread_start_us":125,"threads_started":1}
I20260812 04:01:34.046624 20890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d78d8e31b4417ea7e95a5767d8a0f2" candidate_uuid: "aee481b2b30f4276bd84302ec8e06e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e98f4a399dd40da8db0c44bac9424fd" is_pre_election: true
I20260812 04:01:34.046833 20890 raft_consensus.cc:2454] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aee481b2b30f4276bd84302ec8e06e41 in term 0.
I20260812 04:01:34.046838 21179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d78d8e31b4417ea7e95a5767d8a0f2" candidate_uuid: "aee481b2b30f4276bd84302ec8e06e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cf3c2c39f484e81956e2b5a6c473b15" is_pre_election: true
I20260812 04:01:34.047019 21179 raft_consensus.cc:2454] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aee481b2b30f4276bd84302ec8e06e41 in term 0.
I20260812 04:01:34.047156 20966 leader_election.cc:302] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e98f4a399dd40da8db0c44bac9424fd, aee481b2b30f4276bd84302ec8e06e41; no voters:
I20260812 04:01:34.047312 21257 raft_consensus.cc:2781] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 04:01:34.047380 21257 raft_consensus.cc:482] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 04:01:34.047415 21257 raft_consensus.cc:3037] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:34.048182 21257 raft_consensus.cc:504] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee481b2b30f4276bd84302ec8e06e41" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 44139 } } peers { permanent_uuid: "4cf3c2c39f484e81956e2b5a6c473b15" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46651 } } peers { permanent_uuid: "0e98f4a399dd40da8db0c44bac9424fd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45541 } }
I20260812 04:01:34.048345 21257 leader_election.cc:288] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [CANDIDATE]: Term 1 election: Requested vote from peers 4cf3c2c39f484e81956e2b5a6c473b15 (127.31.43.195:46651), 0e98f4a399dd40da8db0c44bac9424fd (127.31.43.193:45541)
I20260812 04:01:34.048514 20890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d78d8e31b4417ea7e95a5767d8a0f2" candidate_uuid: "aee481b2b30f4276bd84302ec8e06e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e98f4a399dd40da8db0c44bac9424fd"
I20260812 04:01:34.048504 21179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d78d8e31b4417ea7e95a5767d8a0f2" candidate_uuid: "aee481b2b30f4276bd84302ec8e06e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cf3c2c39f484e81956e2b5a6c473b15"
I20260812 04:01:34.048604 21179 raft_consensus.cc:3037] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:34.048602 20890 raft_consensus.cc:3037] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:34.049556 21179 raft_consensus.cc:2454] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aee481b2b30f4276bd84302ec8e06e41 in term 1.
I20260812 04:01:34.049549 20890 raft_consensus.cc:2454] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aee481b2b30f4276bd84302ec8e06e41 in term 1.
I20260812 04:01:34.049737 20966 leader_election.cc:302] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e98f4a399dd40da8db0c44bac9424fd, aee481b2b30f4276bd84302ec8e06e41; no voters:
I20260812 04:01:34.049844 21257 raft_consensus.cc:2781] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:34.050025 21257 raft_consensus.cc:686] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 1 LEADER]: Becoming Leader. State: Replica: aee481b2b30f4276bd84302ec8e06e41, State: Running, Role: LEADER
I20260812 04:01:34.050163 21257 consensus_queue.cc:237] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee481b2b30f4276bd84302ec8e06e41" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 44139 } } peers { permanent_uuid: "4cf3c2c39f484e81956e2b5a6c473b15" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46651 } } peers { permanent_uuid: "0e98f4a399dd40da8db0c44bac9424fd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45541 } }
I20260812 04:01:34.051138 20736 catalog_manager.cc:5495] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to aee481b2b30f4276bd84302ec8e06e41 (127.31.43.194). New cstate: current_term: 1 leader_uuid: "aee481b2b30f4276bd84302ec8e06e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee481b2b30f4276bd84302ec8e06e41" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 44139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cf3c2c39f484e81956e2b5a6c473b15" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e98f4a399dd40da8db0c44bac9424fd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45541 } health_report { overall_health: UNKNOWN } } }
W20260812 04:01:34.054538 21090 tablet.cc:2369] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:01:34.070979 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:34.068390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.31.43.194:44139)
0812 04:01:34.068466 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:34.068468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:34.068479 (+ 11us) server_negotiation.cc:407] Connection header received
0812 04:01:34.068577 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:34.068578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:34.068599 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:34.068682 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:01:34.068837 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:34.069373 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:34.070158 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:34.070354 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:34.070437 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:34.070584 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:01:34.070707 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:01:34.070822 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:34.070861 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 04:01:34.070871 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 04:01:34.072911 21179 raft_consensus.cc:1261] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15 [term 1 FOLLOWER]: Refusing update from remote peer aee481b2b30f4276bd84302ec8e06e41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:01:34.072964 20890 raft_consensus.cc:1261] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd [term 1 FOLLOWER]: Refusing update from remote peer aee481b2b30f4276bd84302ec8e06e41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:01:34.073313 21260 consensus_queue.cc:1035] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cf3c2c39f484e81956e2b5a6c473b15" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 46651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:01:34.073441 21257 consensus_queue.cc:1035] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e98f4a399dd40da8db0c44bac9424fd" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 45541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:01:34.075981 21263 mvcc.cc:205] Tried to move back new op lower bound from 7317533876519657472 to 7317533876430184448. Current Snapshot: MvccSnapshot[applied={T|T < 7317533876519657472}]
I20260812 04:01:34.076632 21266 mvcc.cc:205] Tried to move back new op lower bound from 7317533876519657472 to 7317533876430184448. Current Snapshot: MvccSnapshot[applied={T|T < 7317533876519657472}]
I20260812 04:01:34.078788 20736 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:01:34.078910 20736 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:01:34.079723 20736 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87b5a2a1be8e42a88527164f4058283f: Sending DeleteTablet for 3 replicas of tablet 33d78d8e31b4417ea7e95a5767d8a0f2
I20260812 04:01:34.080145 21158 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d78d8e31b4417ea7e95a5767d8a0f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:34 UTC) from {username='slave'} at 127.0.0.1:59742
I20260812 04:01:34.080129 20868 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d78d8e31b4417ea7e95a5767d8a0f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:34 UTC) from {username='slave'} at 127.0.0.1:35090
I20260812 04:01:34.080403 21274 tablet_replica.cc:330] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd: stopping tablet replica
I20260812 04:01:34.080521 21274 raft_consensus.cc:2229] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:01:34.080565 21013 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d78d8e31b4417ea7e95a5767d8a0f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:34 UTC) from {username='slave'} at 127.0.0.1:34738
I20260812 04:01:34.080633 21274 raft_consensus.cc:2258] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 0e98f4a399dd40da8db0c44bac9424fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:34.080780 21276 tablet_replica.cc:330] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41: stopping tablet replica
I20260812 04:01:34.080804 21275 tablet_replica.cc:330] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15: stopping tablet replica
I20260812 04:01:34.080910 21275 raft_consensus.cc:2229] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:01:34.080921 21276 raft_consensus.cc:2229] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 1 LEADER]: Raft consensus shutting down.
I20260812 04:01:34.080996 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 20803
I20260812 04:01:34.081010 21275 raft_consensus.cc:2258] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:34.081161 21276 raft_consensus.cc:2258] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:34.081501 21275 ts_tablet_manager.cc:1905] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:01:34.081722 21276 ts_tablet_manager.cc:1905] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:01:34.083297 21275 ts_tablet_manager.cc:1918] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:01:34.083384 21275 log.cc:1196] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/33d78d8e31b4417ea7e95a5767d8a0f2
I20260812 04:01:34.083664 21275 ts_tablet_manager.cc:1939] T 33d78d8e31b4417ea7e95a5767d8a0f2 P 4cf3c2c39f484e81956e2b5a6c473b15: Deleting consensus metadata
I20260812 04:01:34.084378 21276 ts_tablet_manager.cc:1918] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:01:34.084534 21276 log.cc:1196] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/33d78d8e31b4417ea7e95a5767d8a0f2
I20260812 04:01:34.084554 20722 catalog_manager.cc:4841] TS 4cf3c2c39f484e81956e2b5a6c473b15 (127.31.43.195:46651): tablet 33d78d8e31b4417ea7e95a5767d8a0f2 (table test-table [id=4c5d61d7175049fb88bb12825314bf47]) successfully deleted
I20260812 04:01:34.084944 21276 ts_tablet_manager.cc:1939] T 33d78d8e31b4417ea7e95a5767d8a0f2 P aee481b2b30f4276bd84302ec8e06e41: Deleting consensus metadata
I20260812 04:01:34.087507 20721 catalog_manager.cc:4841] TS aee481b2b30f4276bd84302ec8e06e41 (127.31.43.194:44139): tablet 33d78d8e31b4417ea7e95a5767d8a0f2 (table test-table [id=4c5d61d7175049fb88bb12825314bf47]) successfully deleted
W20260812 04:01:34.093660 20723 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:45541 (error 108)
I20260812 04:01:34.094054 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 20947
I20260812 04:01:34.094303 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:34.094148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:45541 (local address 127.0.0.1:35096)
0812 04:01:34.094210 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:34.094263 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:45541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 04:01:34.094431 20723 catalog_manager.cc:4584] TS 0e98f4a399dd40da8db0c44bac9424fd (127.31.43.193:45541): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33d78d8e31b4417ea7e95a5767d8a0f2: Network error: Client connection negotiation failed: client connection to 127.31.43.193:45541: connect: Connection refused (error 111)
I20260812 04:01:34.101195 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 21092
I20260812 04:01:34.109648 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 20704
2026-08-12T04:01:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (750 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:34 dist-test-slave-8tdl krb5kdc[21281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:34 dist-test-slave-8tdl krb5kdc[21281](info): set up 2 sockets
Aug 12 04:01:34 dist-test-slave-8tdl krb5kdc[21281](info): commencing operation
krb5kdc: starting...
W20260812 04:01:35.240087 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:35 dist-test-slave-8tdl krb5kdc[21281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:01:35.382295 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:43561
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:43561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:35.472070 21295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:35.472301 21295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:35.472326 21295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:35.473526 21295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:35.473574 21295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:35.473588 21295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:35.473603 21295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:35.473614 21295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:35.475236 21295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:43561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:43561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:35.475513 21295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:35.475786 21295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:35.481020 21302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:35.481065 21301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:35.481036 21305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:35.481151 21295 server_base.cc:1034] running on GCE node
I20260812 04:01:35.481600 21295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:35.481921 21295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:35.483095 21295 hybrid_clock.cc:648] HybridClock initialized: now 1786507295483079 us; error 45 us; skew 500 ppm
Aug 12 04:01:35 dist-test-slave-8tdl krb5kdc[21281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507295, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:35.485757 21295 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:01:35.486318 21295 webserver.cc:460] Webserver started at http://127.31.43.254:41723/ using document root <none> and password file <none>
I20260812 04:01:35.486572 21295 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:35.486641 21295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:35.486845 21295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:35.487932 21295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "cff656d1169c42ef8c1b9fedd39169b7"
format_stamp: "Formatted at 2026-08-12 04:01:35 on dist-test-slave-8tdl"
server_key: "6e5c7d658c0493b52599afd8cadf3ddf"
server_key_iv: "6424f89057ba6fb465354bc7edaf336f"
server_key_version: "encryptionkey@0"
I20260812 04:01:35.488327 21295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "cff656d1169c42ef8c1b9fedd39169b7"
format_stamp: "Formatted at 2026-08-12 04:01:35 on dist-test-slave-8tdl"
server_key: "6e5c7d658c0493b52599afd8cadf3ddf"
server_key_iv: "6424f89057ba6fb465354bc7edaf336f"
server_key_version: "encryptionkey@0"
I20260812 04:01:35.489696 21295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:35.490496 21312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:35.490707 21295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:35.490818 21295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "cff656d1169c42ef8c1b9fedd39169b7"
format_stamp: "Formatted at 2026-08-12 04:01:35 on dist-test-slave-8tdl"
server_key: "6e5c7d658c0493b52599afd8cadf3ddf"
server_key_iv: "6424f89057ba6fb465354bc7edaf336f"
server_key_version: "encryptionkey@0"
I20260812 04:01:35.490900 21295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:35.528966 21295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:35.531437 21295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:35.531664 21295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:35.536165 21295 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:43561
I20260812 04:01:35.536221 21370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:43561 every 8 connection(s)
I20260812 04:01:35.536710 21295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:35.537654 21371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:35.539837 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 21295
I20260812 04:01:35.539987 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:35.540196 31919 external_mini_cluster.cc:1428] Setting key 4476574fa62eb99f0fb385f2e0f517f5
I20260812 04:01:35.540514 21371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7: Bootstrap starting.
I20260812 04:01:35.541218 21371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:35.541525 21371 log.cc:824] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:35.542253 21371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7: No bootstrap required, opened a new log
I20260812 04:01:35.544224 21371 raft_consensus.cc:348] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff656d1169c42ef8c1b9fedd39169b7" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43561 } }
I20260812 04:01:35.544399 21371 raft_consensus.cc:374] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:35.544448 21371 raft_consensus.cc:729] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff656d1169c42ef8c1b9fedd39169b7, State: Initialized, Role: FOLLOWER
I20260812 04:01:35.544631 21371 consensus_queue.cc:260] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff656d1169c42ef8c1b9fedd39169b7" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43561 } }
I20260812 04:01:35.544730 21371 raft_consensus.cc:388] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:35.544790 21371 raft_consensus.cc:482] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:35.544843 21371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:35.545599 21371 raft_consensus.cc:504] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff656d1169c42ef8c1b9fedd39169b7" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43561 } }
I20260812 04:01:35.545732 21371 leader_election.cc:302] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cff656d1169c42ef8c1b9fedd39169b7; no voters:
I20260812 04:01:35.545966 21371 leader_election.cc:288] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:35.545984 21379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:35.546170 21379 raft_consensus.cc:686] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [term 1 LEADER]: Becoming Leader. State: Replica: cff656d1169c42ef8c1b9fedd39169b7, State: Running, Role: LEADER
I20260812 04:01:35.546295 21371 sys_catalog.cc:564] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:35.546316 21379 consensus_queue.cc:237] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff656d1169c42ef8c1b9fedd39169b7" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43561 } }
I20260812 04:01:35.546845 21381 sys_catalog.cc:455] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cff656d1169c42ef8c1b9fedd39169b7. Latest consensus state: current_term: 1 leader_uuid: "cff656d1169c42ef8c1b9fedd39169b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff656d1169c42ef8c1b9fedd39169b7" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43561 } } }
I20260812 04:01:35.547009 21381 sys_catalog.cc:458] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [sys.catalog]: This master's current role is: LEADER
Aug 12 04:01:35 dist-test-slave-8tdl krb5kdc[21281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:35.547475 21380 sys_catalog.cc:455] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cff656d1169c42ef8c1b9fedd39169b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff656d1169c42ef8c1b9fedd39169b7" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43561 } } }
I20260812 04:01:35.547576 21380 sys_catalog.cc:458] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:35.547971 21393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:35.548527 21393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:35.551136 21393 catalog_manager.cc:1347] Generated new cluster ID: 817224c1fb7348b68da62c3967de335e
I20260812 04:01:35.551196 21393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:35.552881 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:35.540725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53606 (local address 127.31.43.254:43561)
0812 04:01:35.540916 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:35.540927 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:35.540964 (+ 37us) server_negotiation.cc:407] Connection header received
0812 04:01:35.541481 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:35.541489 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:35.541540 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:35.541726 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:35.542415 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:35.543362 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:35.544127 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:35.544648 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:35.544782 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:35.547772 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:35.547776 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:35.547776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:35.547788 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:35.551063 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:35.551503 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:35.551506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:35.551507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:35.551613 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:35.551854 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:35.551856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:35.551856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:35.552234 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:35.552404 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:35.552529 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 04:01:35.552666 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260812 04:01:35.569128 21393 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:35.569676 21393 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:35.573515 21393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7: Generated new TSK 0
I20260812 04:01:35.573720 21393 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.43.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
Entry for principal kudu/127.31.43.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab.
I20260812 04:01:35.610289 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43561
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:35.698053 21402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:35.698289 21402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:35.698328 21402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:35.699566 21402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:35.699635 21402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:35.699679 21402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:01:35.701236 21402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.193.keytab
--principal=kudu/127.31.43.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:35.701519 21402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:35.701766 21402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:35.707123 21408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:35.707144 21407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:35.707203 21410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:35.707412 21402 server_base.cc:1034] running on GCE node
I20260812 04:01:35.707613 21402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:35.707816 21402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:35.708952 21402 hybrid_clock.cc:648] HybridClock initialized: now 1786507295708941 us; error 31 us; skew 500 ppm
Aug 12 04:01:35 dist-test-slave-8tdl krb5kdc[21281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507295, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:35.711745 21402 init.cc:377] Logged in from keytab as kudu/127.31.43.193@KRBTEST.COM (short username kudu)
I20260812 04:01:35.712250 21402 webserver.cc:460] Webserver started at http://127.31.43.193:39055/ using document root <none> and password file <none>
I20260812 04:01:35.712503 21402 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:35.712553 21402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:35.712728 21402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:35.713794 21402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "a187286152504595bb8eb1e968ccce87"
format_stamp: "Formatted at 2026-08-12 04:01:35 on dist-test-slave-8tdl"
server_key: "3d12733d3a1591dbb7e4b660bbdd8ca8"
server_key_iv: "1d013299aed51b25221824d449aa214d"
server_key_version: "encryptionkey@0"
I20260812 04:01:35.714195 21402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "a187286152504595bb8eb1e968ccce87"
format_stamp: "Formatted at 2026-08-12 04:01:35 on dist-test-slave-8tdl"
server_key: "3d12733d3a1591dbb7e4b660bbdd8ca8"
server_key_iv: "1d013299aed51b25221824d449aa214d"
server_key_version: "encryptionkey@0"
I20260812 04:01:35.716241 21402 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 04:01:35.717204 21417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:35.717398 21402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:35.717496 21402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "a187286152504595bb8eb1e968ccce87"
format_stamp: "Formatted at 2026-08-12 04:01:35 on dist-test-slave-8tdl"
server_key: "3d12733d3a1591dbb7e4b660bbdd8ca8"
server_key_iv: "1d013299aed51b25221824d449aa214d"
server_key_version: "encryptionkey@0"
I20260812 04:01:35.717571 21402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:35.735126 21402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:35.737764 21402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:35.737948 21402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:35.738222 21402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:35.738597 21402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:35.738653 21402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:35.738713 21402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:35.738749 21402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:35.745203 21402 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:40925
I20260812 04:01:35.745256 21541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:40925 every 8 connection(s)
I20260812 04:01:35.745599 21402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 04:01:35.746789 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 21402
I20260812 04:01:35.746907 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 04:01:35.747174 31919 external_mini_cluster.cc:1428] Setting key 17385917103fbbf19dce9c4a91f7a682
Aug 12 04:01:35 dist-test-slave-8tdl krb5kdc[21281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507295, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.193@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:35.756775 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:35.746440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:49051 (local address 127.31.43.254:43561)
0812 04:01:35.746521 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:35.746523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:35.746800 (+ 277us) server_negotiation.cc:407] Connection header received
0812 04:01:35.747584 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:35.747585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:35.747598 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:35.747665 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:35.748846 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:35.749621 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:35.750717 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:35.750932 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:35.751037 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:35.753686 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:35.753688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:35.753688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:35.753696 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:35.755327 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:35.755753 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:35.755755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:35.755755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:35.755815 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:35.756087 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:35.756088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:35.756088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:35.756213 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:35.756291 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:35.756583 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 04:01:35.756680 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
WARNING: no policy specified for kudu/127.31.43.194@KRBTEST.COM; defaulting to no policy
I20260812 04:01:35.756992 21544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:35.746403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:43561 (local address 127.31.43.193:49051)
0812 04:01:35.746612 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:35.746670 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:35.747458 (+ 788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:35.747758 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:35.747762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:35.748537 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:35.748743 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:35.748748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:35.749723 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:35.749724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:35.750570 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:35.750572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:35.751058 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:35.751059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:35.751217 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:35.751832 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:01:35.751847 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:35.753569 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:35.755420 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:35.755428 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:35.755431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:35.755669 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:35.755885 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:35.755886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:35.755886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:35.756015 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:35.756282 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:35.756290 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:35.756441 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 04:01:35.756560 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 04:01:35.756727 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260812 04:01:35.757545 21542 heartbeater.cc:347] Connected to a master server at 127.31.43.254:43561
I20260812 04:01:35.757673 21542 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:35.757861 21542 heartbeater.cc:511] Master 127.31.43.254:43561 requested a full tablet report, sending...
I20260812 04:01:35.758524 21330 ts_manager.cc:194] Registered new tserver with Master: a187286152504595bb8eb1e968ccce87 (127.31.43.193:40925)
I20260812 04:01:35.759203 21330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.193@KRBTEST.COM'} at 127.31.43.193:49051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
Entry for principal kudu/127.31.43.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab.
I20260812 04:01:35.777936 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43561
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:35.866477 21549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:35.866741 21549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:35.866768 21549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:35.867935 21549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:35.867978 21549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:35.868005 21549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:01:35.869516 21549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.194.keytab
--principal=kudu/127.31.43.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:35.869735 21549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:35.869930 21549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:35.875305 21554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:35.875483 21549 server_base.cc:1034] running on GCE node
W20260812 04:01:35.875432 21557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:35.875499 21555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:35.875849 21549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:35.876151 21549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:35.877301 21549 hybrid_clock.cc:648] HybridClock initialized: now 1786507295877282 us; error 42 us; skew 500 ppm
Aug 12 04:01:35 dist-test-slave-8tdl krb5kdc[21281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507295, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:35.880067 21549 init.cc:377] Logged in from keytab as kudu/127.31.43.194@KRBTEST.COM (short username kudu)
I20260812 04:01:35.880623 21549 webserver.cc:460] Webserver started at http://127.31.43.194:40699/ using document root <none> and password file <none>
I20260812 04:01:35.880867 21549 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:35.880940 21549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:35.881119 21549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:35.882249 21549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "dece4c1385c14922972ca4449acce461"
format_stamp: "Formatted at 2026-08-12 04:01:35 on dist-test-slave-8tdl"
server_key: "c152ab48540ab11e34e0da6491f503f1"
server_key_iv: "731d7ac647297308adebdbfb8c6ac75d"
server_key_version: "encryptionkey@0"
I20260812 04:01:35.882669 21549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "dece4c1385c14922972ca4449acce461"
format_stamp: "Formatted at 2026-08-12 04:01:35 on dist-test-slave-8tdl"
server_key: "c152ab48540ab11e34e0da6491f503f1"
server_key_iv: "731d7ac647297308adebdbfb8c6ac75d"
server_key_version: "encryptionkey@0"
I20260812 04:01:35.884218 21549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:35.885015 21564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:35.885246 21549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:35.885362 21549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "dece4c1385c14922972ca4449acce461"
format_stamp: "Formatted at 2026-08-12 04:01:35 on dist-test-slave-8tdl"
server_key: "c152ab48540ab11e34e0da6491f503f1"
server_key_iv: "731d7ac647297308adebdbfb8c6ac75d"
server_key_version: "encryptionkey@0"
I20260812 04:01:35.885458 21549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:35.898648 21549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:35.900799 21549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:35.900979 21549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:35.901233 21549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:35.901571 21549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:35.901638 21549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:35.901677 21549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:35.901722 21549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:35.907940 21549 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:37361
I20260812 04:01:35.907997 21685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:37361 every 8 connection(s)
I20260812 04:01:35.908380 21549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:01:35.914338 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 21549
I20260812 04:01:35.914444 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 04:01:35.914691 31919 external_mini_cluster.cc:1428] Setting key eb7881627e209b341ecaf04ebbdf29db
Aug 12 04:01:35 dist-test-slave-8tdl krb5kdc[21281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507295, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.194@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:35.919289 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:35.909142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:50459 (local address 127.31.43.254:43561)
0812 04:01:35.909221 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:35.909223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:35.909484 (+ 261us) server_negotiation.cc:407] Connection header received
0812 04:01:35.909982 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:35.909983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:35.909994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:35.910049 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:35.911150 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:35.911654 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:35.912758 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:35.913017 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:35.913107 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:35.916047 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:35.916049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:35.916049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:35.916058 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:35.917786 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:35.918220 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:35.918221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:35.918222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:35.918289 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:35.918554 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:35.918555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:35.918555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:35.918732 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:35.918799 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:35.919096 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 04:01:35.919199 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 04:01:35.919485 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:35.909123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:43561 (local address 127.31.43.194:50459)
0812 04:01:35.909321 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:35.909378 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:35.909886 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:35.910113 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:35.910116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:35.910837 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:35.911051 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:35.911057 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:35.911740 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:35.911741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:35.912575 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:35.912578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:35.913210 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:35.913211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:35.913332 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:35.914024 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:01:35.914032 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:35.915945 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:35.917877 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:35.917886 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:35.917889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:35.918132 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:35.918361 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:35.918362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:35.918362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:35.918481 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:35.918790 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:35.918798 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:35.918946 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 04:01:35.919058 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 04:01:35.919230 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":112,"threads_started":1}
I20260812 04:01:35.920075 21686 heartbeater.cc:347] Connected to a master server at 127.31.43.254:43561
I20260812 04:01:35.920197 21686 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:35.920423 21686 heartbeater.cc:511] Master 127.31.43.254:43561 requested a full tablet report, sending...
I20260812 04:01:35.920858 21330 ts_manager.cc:194] Registered new tserver with Master: dece4c1385c14922972ca4449acce461 (127.31.43.194:37361)
I20260812 04:01:35.921243 21330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.194@KRBTEST.COM'} at 127.31.43.194:50459
WARNING: no policy specified for kudu/127.31.43.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
Entry for principal kudu/127.31.43.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab.
I20260812 04:01:35.945415 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43561
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:36.031947 21691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:36.032167 21691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:36.032192 21691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:36.033280 21691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:36.033319 21691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:36.033346 21691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:01:36.034809 21691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.195.keytab
--principal=kudu/127.31.43.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:43561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:36.035024 21691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:36.035226 21691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:36.040642 21697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:36.040630 21698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:36.040630 21700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:36.040939 21691 server_base.cc:1034] running on GCE node
I20260812 04:01:36.041152 21691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:36.041385 21691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:36.042527 21691 hybrid_clock.cc:648] HybridClock initialized: now 1786507296042510 us; error 33 us; skew 500 ppm
Aug 12 04:01:36 dist-test-slave-8tdl krb5kdc[21281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507296, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:36.045171 21691 init.cc:377] Logged in from keytab as kudu/127.31.43.195@KRBTEST.COM (short username kudu)
I20260812 04:01:36.045665 21691 webserver.cc:460] Webserver started at http://127.31.43.195:35359/ using document root <none> and password file <none>
I20260812 04:01:36.046005 21691 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:36.046058 21691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:36.046257 21691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:36.047652 21691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "485350ea585f48769267f57e0bb46f3c"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "68973df235dde76452168e0c37340e55"
server_key_iv: "78bb52863d48157c24e35eafa1e22d7d"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.048050 21691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "485350ea585f48769267f57e0bb46f3c"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "68973df235dde76452168e0c37340e55"
server_key_iv: "78bb52863d48157c24e35eafa1e22d7d"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.049537 21691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:36.050371 21708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.050565 21691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:36.050704 21691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "485350ea585f48769267f57e0bb46f3c"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "68973df235dde76452168e0c37340e55"
server_key_iv: "78bb52863d48157c24e35eafa1e22d7d"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.050797 21691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:36.082451 21691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:36.084587 21691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:36.084765 21691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:36.085031 21691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:36.085379 21691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:36.085438 21691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.085482 21691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:36.085507 21691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.092763 21691 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:44687
I20260812 04:01:36.092825 21832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:44687 every 8 connection(s)
I20260812 04:01:36.093211 21691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
Aug 12 04:01:36 dist-test-slave-8tdl krb5kdc[21281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507296, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.195@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:36.102150 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 21691
I20260812 04:01:36.102265 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 04:01:36.102459 31919 external_mini_cluster.cc:1428] Setting key 42bd17d81ff7cd4e783ca4261d1e247f
I20260812 04:01:36.103866 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.093956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:39373 (local address 127.31.43.254:43561)
0812 04:01:36.094038 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.094040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.094278 (+ 238us) server_negotiation.cc:407] Connection header received
0812 04:01:36.094998 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.095000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.095014 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.095074 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:36.096206 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.096786 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.097911 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.098138 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.098235 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.100570 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:36.100572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:36.100572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:36.100653 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:36.102069 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:36.102640 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:36.102641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:36.102642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:36.102759 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:36.103126 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:36.103127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:36.103127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:36.103288 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:36.103360 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.103710 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.103799 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 04:01:36.104085 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.093956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:43561 (local address 127.31.43.195:39373)
0812 04:01:36.094139 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.094178 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.094848 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.095153 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.095157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.095848 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:36.096078 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.096083 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.096895 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.096896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.097791 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.097793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.098248 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.098249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.098385 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.098922 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:01:36.098930 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:36.100454 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:36.102190 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:36.102199 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:36.102202 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:36.102550 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:36.102839 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:01:36.102840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:01:36.102840 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:01:36.103039 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:01:36.103368 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:36.103376 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:36.103554 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.103673 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.103835 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":111,"threads_started":1}
I20260812 04:01:36.104511 21833 heartbeater.cc:347] Connected to a master server at 127.31.43.254:43561
I20260812 04:01:36.104609 21833 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:36.104764 21833 heartbeater.cc:511] Master 127.31.43.254:43561 requested a full tablet report, sending...
I20260812 04:01:36.105141 21330 ts_manager.cc:194] Registered new tserver with Master: 485350ea585f48769267f57e0bb46f3c (127.31.43.195:44687)
I20260812 04:01:36.105526 21330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.43.195@KRBTEST.COM'} at 127.31.43.195:39373
I20260812 04:01:36.114481 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 04:01:36.122330 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.116022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53616 (local address 127.31.43.254:43561)
0812 04:01:36.116114 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.116116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.116220 (+ 104us) server_negotiation.cc:407] Connection header received
0812 04:01:36.116261 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.116262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.116271 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.116325 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:36.116906 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.117444 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.118287 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.118472 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.118525 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.119881 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:36.119882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:36.119882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:36.119888 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:36.121220 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:36.121602 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:36.121603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:36.121603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:36.121657 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:36.121845 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:36.121846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:36.121846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:36.122006 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:36.122104 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.122196 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.122262 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 04:01:36.124078 21330 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 04:01:36.125680 21330 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 04:01:36.132550 21847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.128060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:44687 (local address 127.0.0.1:54292)
0812 04:01:36.128361 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.128409 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.128483 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.128800 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.128803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.128806 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:36.128887 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.128889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.130042 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.130043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.131407 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.131410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.132343 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.132344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.132406 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.132408 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.132459 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.132491 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":102,"threads_started":1}
I20260812 04:01:36.132551 21849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.128061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:37361 (local address 127.0.0.1:59112)
0812 04:01:36.128500 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.128509 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.128551 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.128919 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.128920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.128923 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:36.129005 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.129007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.130340 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.130341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.131414 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.131416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.132209 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.132210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.132297 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.132312 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.132385 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.132446 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":1,"thread_start_us":53,"threads_started":1}
I20260812 04:01:36.132794 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.128307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59112 (local address 127.31.43.194:37361)
0812 04:01:36.128676 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.128680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.128688 (+ 8us) server_negotiation.cc:407] Connection header received
0812 04:01:36.128720 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.128741 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.128758 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.128834 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:36.129079 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.130226 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.131507 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.132111 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.132206 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.132272 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.132471 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.132531 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":93,"threads_started":1}
I20260812 04:01:36.132858 21850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.128276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54292 (local address 127.31.43.195:44687)
0812 04:01:36.128607 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.128611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.128623 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:01:36.128685 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.128688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.128704 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.128793 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:36.129070 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.129933 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.131538 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.132266 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.132347 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.132400 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.132697 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.132747 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":116,"threads_started":1}
I20260812 04:01:36.133373 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.128070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:40925 (local address 127.0.0.1:43856)
0812 04:01:36.129111 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.129123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.129172 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.129361 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.129362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.129364 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:36.129403 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.129405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.130586 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.130587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.131646 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.131649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.132659 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.133116 (+ 457us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.133195 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.133201 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.133262 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.133305 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"thread_start_us":119,"threads_started":1}
I20260812 04:01:36.133630 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.128377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.31.43.193:40925)
0812 04:01:36.128541 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.128547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.129194 (+ 647us) server_negotiation.cc:407] Connection header received
0812 04:01:36.129243 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.129245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.129262 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.129347 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:36.129464 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.130475 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.131738 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.132467 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.132534 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.132584 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.133485 (+ 901us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.133538 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260812 04:01:36.133595 21760 tablet_service.cc:1467] Processing CreateTablet for tablet 91908decc82d4cf08c98adbf3869d598 (DEFAULT_TABLE table=test-table [id=1c2e90398ae44e78bb8ae05996eeb02a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:36.133603 21614 tablet_service.cc:1467] Processing CreateTablet for tablet 91908decc82d4cf08c98adbf3869d598 (DEFAULT_TABLE table=test-table [id=1c2e90398ae44e78bb8ae05996eeb02a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:36.133989 21760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91908decc82d4cf08c98adbf3869d598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:36.133991 21614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91908decc82d4cf08c98adbf3869d598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:36.136979 21853 tablet_bootstrap.cc:492] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461: Bootstrap starting.
I20260812 04:01:36.137733 21853 tablet_bootstrap.cc:654] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:36.137704 21468 tablet_service.cc:1467] Processing CreateTablet for tablet 91908decc82d4cf08c98adbf3869d598 (DEFAULT_TABLE table=test-table [id=1c2e90398ae44e78bb8ae05996eeb02a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:36.138109 21468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91908decc82d4cf08c98adbf3869d598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:36.138304 21853 log.cc:824] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:36.138716 21854 tablet_bootstrap.cc:492] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c: Bootstrap starting.
I20260812 04:01:36.139292 21854 tablet_bootstrap.cc:654] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:36.140085 21853 tablet_bootstrap.cc:492] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461: No bootstrap required, opened a new log
I20260812 04:01:36.140144 21853 ts_tablet_manager.cc:1397] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 04:01:36.141198 21856 tablet_bootstrap.cc:492] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87: Bootstrap starting.
I20260812 04:01:36.141921 21856 tablet_bootstrap.cc:654] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:36.141837 21853 raft_consensus.cc:348] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dece4c1385c14922972ca4449acce461" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37361 } } peers { permanent_uuid: "485350ea585f48769267f57e0bb46f3c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44687 } } peers { permanent_uuid: "a187286152504595bb8eb1e968ccce87" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40925 } }
I20260812 04:01:36.142027 21853 raft_consensus.cc:374] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:36.142095 21853 raft_consensus.cc:729] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dece4c1385c14922972ca4449acce461, State: Initialized, Role: FOLLOWER
I20260812 04:01:36.142251 21853 consensus_queue.cc:260] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dece4c1385c14922972ca4449acce461" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37361 } } peers { permanent_uuid: "485350ea585f48769267f57e0bb46f3c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44687 } } peers { permanent_uuid: "a187286152504595bb8eb1e968ccce87" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40925 } }
I20260812 04:01:36.142541 21856 log.cc:824] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:36.142573 21853 ts_tablet_manager.cc:1428] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 04:01:36.142632 21686 heartbeater.cc:503] Master 127.31.43.254:43561 was elected leader, sending a full tablet report...
I20260812 04:01:36.142956 21854 log.cc:824] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:36.143393 21856 tablet_bootstrap.cc:492] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87: No bootstrap required, opened a new log
I20260812 04:01:36.143476 21856 ts_tablet_manager.cc:1397] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 04:01:36.144163 21854 tablet_bootstrap.cc:492] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c: No bootstrap required, opened a new log
I20260812 04:01:36.144237 21854 ts_tablet_manager.cc:1397] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 04:01:36.145573 21856 raft_consensus.cc:348] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dece4c1385c14922972ca4449acce461" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37361 } } peers { permanent_uuid: "485350ea585f48769267f57e0bb46f3c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44687 } } peers { permanent_uuid: "a187286152504595bb8eb1e968ccce87" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40925 } }
I20260812 04:01:36.145771 21856 raft_consensus.cc:374] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:36.145807 21856 raft_consensus.cc:729] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a187286152504595bb8eb1e968ccce87, State: Initialized, Role: FOLLOWER
I20260812 04:01:36.145929 21856 consensus_queue.cc:260] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dece4c1385c14922972ca4449acce461" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37361 } } peers { permanent_uuid: "485350ea585f48769267f57e0bb46f3c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44687 } } peers { permanent_uuid: "a187286152504595bb8eb1e968ccce87" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40925 } }
I20260812 04:01:36.146217 21856 ts_tablet_manager.cc:1428] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 04:01:36.146241 21542 heartbeater.cc:503] Master 127.31.43.254:43561 was elected leader, sending a full tablet report...
I20260812 04:01:36.146291 21854 raft_consensus.cc:348] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dece4c1385c14922972ca4449acce461" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37361 } } peers { permanent_uuid: "485350ea585f48769267f57e0bb46f3c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44687 } } peers { permanent_uuid: "a187286152504595bb8eb1e968ccce87" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40925 } }
I20260812 04:01:36.146445 21854 raft_consensus.cc:374] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:36.146520 21854 raft_consensus.cc:729] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 485350ea585f48769267f57e0bb46f3c, State: Initialized, Role: FOLLOWER
I20260812 04:01:36.146644 21854 consensus_queue.cc:260] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dece4c1385c14922972ca4449acce461" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37361 } } peers { permanent_uuid: "485350ea585f48769267f57e0bb46f3c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44687 } } peers { permanent_uuid: "a187286152504595bb8eb1e968ccce87" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40925 } }
I20260812 04:01:36.146962 21854 ts_tablet_manager.cc:1428] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 04:01:36.147006 21833 heartbeater.cc:503] Master 127.31.43.254:43561 was elected leader, sending a full tablet report...
W20260812 04:01:36.158957 21687 tablet.cc:2369] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:01:36.223793 21858 raft_consensus.cc:482] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:01:36.223928 21858 raft_consensus.cc:504] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dece4c1385c14922972ca4449acce461" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37361 } } peers { permanent_uuid: "485350ea585f48769267f57e0bb46f3c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44687 } } peers { permanent_uuid: "a187286152504595bb8eb1e968ccce87" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40925 } }
I20260812 04:01:36.224292 21858 leader_election.cc:288] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 485350ea585f48769267f57e0bb46f3c (127.31.43.195:44687), a187286152504595bb8eb1e968ccce87 (127.31.43.193:40925)
I20260812 04:01:36.227274 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.224444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:44687 (local address 127.31.43.194:57223)
0812 04:01:36.224485 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.224498 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.224554 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.224802 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.224803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.224804 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:36.224886 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.224888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.225605 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.225606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.226352 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.226354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.227000 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.227001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.227085 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.227088 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.227156 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.227198 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 04:01:36.227274 21850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.224614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:57223 (local address 127.31.43.195:44687)
0812 04:01:36.224663 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.224665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.224674 (+ 9us) server_negotiation.cc:407] Connection header received
0812 04:01:36.224703 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.224704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.224718 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.224795 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:36.224968 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.225475 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.226453 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.226913 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.227000 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.227010 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.227162 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.227200 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 04:01:36.227660 21781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91908decc82d4cf08c98adbf3869d598" candidate_uuid: "dece4c1385c14922972ca4449acce461" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "485350ea585f48769267f57e0bb46f3c" is_pre_election: true
I20260812 04:01:36.227803 21863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.224511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:40925 (local address 127.31.43.194:53789)
0812 04:01:36.224735 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.224745 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.224832 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.225063 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.225064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.225066 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:36.225148 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.225149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.226069 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.226070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.226902 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.226905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.227549 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.227550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.227611 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.227614 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.227685 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.227722 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":149,"threads_started":1}
I20260812 04:01:36.227799 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.224606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:53789 (local address 127.31.43.193:40925)
0812 04:01:36.224663 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.224666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.224821 (+ 155us) server_negotiation.cc:407] Connection header received
0812 04:01:36.224900 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.224902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.224911 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.225055 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:36.225231 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.225896 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.227006 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.227424 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.227500 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.227509 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.227692 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.227736 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 04:01:36.227849 21781 raft_consensus.cc:2454] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dece4c1385c14922972ca4449acce461 in term 0.
I20260812 04:01:36.228150 21568 leader_election.cc:302] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 485350ea585f48769267f57e0bb46f3c, dece4c1385c14922972ca4449acce461; no voters:
I20260812 04:01:36.228329 21858 raft_consensus.cc:2781] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 04:01:36.228300 21490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91908decc82d4cf08c98adbf3869d598" candidate_uuid: "dece4c1385c14922972ca4449acce461" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a187286152504595bb8eb1e968ccce87" is_pre_election: true
I20260812 04:01:36.228397 21858 raft_consensus.cc:482] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 04:01:36.228430 21858 raft_consensus.cc:3037] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:36.228504 21490 raft_consensus.cc:2454] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dece4c1385c14922972ca4449acce461 in term 0.
I20260812 04:01:36.229281 21858 raft_consensus.cc:504] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dece4c1385c14922972ca4449acce461" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37361 } } peers { permanent_uuid: "485350ea585f48769267f57e0bb46f3c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44687 } } peers { permanent_uuid: "a187286152504595bb8eb1e968ccce87" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40925 } }
I20260812 04:01:36.229449 21858 leader_election.cc:288] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [CANDIDATE]: Term 1 election: Requested vote from peers 485350ea585f48769267f57e0bb46f3c (127.31.43.195:44687), a187286152504595bb8eb1e968ccce87 (127.31.43.193:40925)
I20260812 04:01:36.229645 21490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91908decc82d4cf08c98adbf3869d598" candidate_uuid: "dece4c1385c14922972ca4449acce461" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a187286152504595bb8eb1e968ccce87"
I20260812 04:01:36.229694 21781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91908decc82d4cf08c98adbf3869d598" candidate_uuid: "dece4c1385c14922972ca4449acce461" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "485350ea585f48769267f57e0bb46f3c"
I20260812 04:01:36.229751 21490 raft_consensus.cc:3037] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:36.229784 21781 raft_consensus.cc:3037] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:36.230515 21781 raft_consensus.cc:2454] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dece4c1385c14922972ca4449acce461 in term 1.
I20260812 04:01:36.230741 21568 leader_election.cc:302] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 485350ea585f48769267f57e0bb46f3c, dece4c1385c14922972ca4449acce461; no voters:
I20260812 04:01:36.230775 21490 raft_consensus.cc:2454] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dece4c1385c14922972ca4449acce461 in term 1.
I20260812 04:01:36.230901 21858 raft_consensus.cc:2781] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:36.231091 21858 raft_consensus.cc:686] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 1 LEADER]: Becoming Leader. State: Replica: dece4c1385c14922972ca4449acce461, State: Running, Role: LEADER
I20260812 04:01:36.231206 21858 consensus_queue.cc:237] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dece4c1385c14922972ca4449acce461" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37361 } } peers { permanent_uuid: "485350ea585f48769267f57e0bb46f3c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44687 } } peers { permanent_uuid: "a187286152504595bb8eb1e968ccce87" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40925 } }
I20260812 04:01:36.232077 21329 catalog_manager.cc:5495] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 reported cstate change: term changed from 0 to 1, leader changed from <none> to dece4c1385c14922972ca4449acce461 (127.31.43.194). New cstate: current_term: 1 leader_uuid: "dece4c1385c14922972ca4449acce461" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dece4c1385c14922972ca4449acce461" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 37361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "485350ea585f48769267f57e0bb46f3c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a187286152504595bb8eb1e968ccce87" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40925 } health_report { overall_health: UNKNOWN } } }
W20260812 04:01:36.246410 21543 tablet.cc:2369] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:01:36.249208 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.246609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59120 (local address 127.31.43.194:37361)
0812 04:01:36.246656 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.246658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.246739 (+ 81us) server_negotiation.cc:407] Connection header received
0812 04:01:36.246786 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.246787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.246805 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.246905 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:01:36.247138 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.247733 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.248483 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.248678 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.248736 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.248883 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:01:36.248990 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:01:36.249097 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.249129 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.249149 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 04:01:36.251186 21781 raft_consensus.cc:1261] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c [term 1 FOLLOWER]: Refusing update from remote peer dece4c1385c14922972ca4449acce461: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:01:36.251217 21490 raft_consensus.cc:1261] T 91908decc82d4cf08c98adbf3869d598 P a187286152504595bb8eb1e968ccce87 [term 1 FOLLOWER]: Refusing update from remote peer dece4c1385c14922972ca4449acce461: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:01:36.251586 21858 consensus_queue.cc:1035] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a187286152504595bb8eb1e968ccce87" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:01:36.251776 21864 consensus_queue.cc:1035] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [LEADER]: Connected to new peer: Peer: permanent_uuid: "485350ea585f48769267f57e0bb46f3c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:01:36.254932 21867 mvcc.cc:205] Tried to move back new op lower bound from 7317533885441658880 to 7317533885363662848. Current Snapshot: MvccSnapshot[applied={T|T < 7317533885441658880}]
I20260812 04:01:36.255097 21869 mvcc.cc:205] Tried to move back new op lower bound from 7317533885441658880 to 7317533885363662848. Current Snapshot: MvccSnapshot[applied={T|T < 7317533885441658880}]
I20260812 04:01:36.255648 21868 mvcc.cc:205] Tried to move back new op lower bound from 7317533885441658880 to 7317533885363662848. Current Snapshot: MvccSnapshot[applied={T|T < 7317533885441658880}]
I20260812 04:01:36.257490 21330 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:01:36.257606 21330 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:01:36.258558 21330 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cff656d1169c42ef8c1b9fedd39169b7: Sending DeleteTablet for 3 replicas of tablet 91908decc82d4cf08c98adbf3869d598
I20260812 04:01:36.259022 21614 tablet_service.cc:1514] Processing DeleteTablet for tablet 91908decc82d4cf08c98adbf3869d598 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:36 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:59112
I20260812 04:01:36.259110 21760 tablet_service.cc:1514] Processing DeleteTablet for tablet 91908decc82d4cf08c98adbf3869d598 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:36 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:54292
I20260812 04:01:36.259254 21468 tablet_service.cc:1514] Processing DeleteTablet for tablet 91908decc82d4cf08c98adbf3869d598 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:36 UTC) from {username='kudu', principal='kudu/127.31.43.254@KRBTEST.COM'} at 127.0.0.1:43856
I20260812 04:01:36.259404 21879 tablet_replica.cc:330] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c: stopping tablet replica
I20260812 04:01:36.259459 21878 tablet_replica.cc:330] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461: stopping tablet replica
I20260812 04:01:36.259567 21878 raft_consensus.cc:2229] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 1 LEADER]: Raft consensus shutting down.
I20260812 04:01:36.259598 21879 raft_consensus.cc:2229] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:01:36.259671 21878 raft_consensus.cc:2258] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:36.259734 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 21402
I20260812 04:01:36.260406 21879 raft_consensus.cc:2258] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:36.261588 21878 ts_tablet_manager.cc:1905] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:01:36.263103 21879 ts_tablet_manager.cc:1905] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:01:36.263716 21878 ts_tablet_manager.cc:1918] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:01:36.263799 21878 log.cc:1196] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/91908decc82d4cf08c98adbf3869d598
I20260812 04:01:36.264107 21878 ts_tablet_manager.cc:1939] T 91908decc82d4cf08c98adbf3869d598 P dece4c1385c14922972ca4449acce461: Deleting consensus metadata
I20260812 04:01:36.264597 21313 catalog_manager.cc:4841] TS dece4c1385c14922972ca4449acce461 (127.31.43.194:37361): tablet 91908decc82d4cf08c98adbf3869d598 (table test-table [id=1c2e90398ae44e78bb8ae05996eeb02a]) successfully deleted
I20260812 04:01:36.265651 21879 ts_tablet_manager.cc:1918] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:01:36.265777 21879 log.cc:1196] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/91908decc82d4cf08c98adbf3869d598
I20260812 04:01:36.266144 21879 ts_tablet_manager.cc:1939] T 91908decc82d4cf08c98adbf3869d598 P 485350ea585f48769267f57e0bb46f3c: Deleting consensus metadata
I20260812 04:01:36.266705 21316 catalog_manager.cc:4841] TS 485350ea585f48769267f57e0bb46f3c (127.31.43.195:44687): tablet 91908decc82d4cf08c98adbf3869d598 (table test-table [id=1c2e90398ae44e78bb8ae05996eeb02a]) successfully deleted
W20260812 04:01:36.270623 21315 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:40925 (error 108)
I20260812 04:01:36.271306 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.271135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:40925 (local address 127.0.0.1:43872)
0812 04:01:36.271209 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.271256 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:40925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 04:01:36.271373 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 21549
W20260812 04:01:36.274964 21315 catalog_manager.cc:4584] TS a187286152504595bb8eb1e968ccce87 (127.31.43.193:40925): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 91908decc82d4cf08c98adbf3869d598: Network error: Client connection negotiation failed: client connection to 127.31.43.193:40925: connect: Connection refused (error 111)
I20260812 04:01:36.280530 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 21691
I20260812 04:01:36.288861 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 21295
2026-08-12T04:01:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2180 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T04:01:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:36Z Disabled control of system clock
I20260812 04:01:36.321798 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:45623
--webserver_interface=127.31.43.254
--webserver_port=0
--builtin_ntp_servers=127.31.43.212:33553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:45623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 04:01:36.408473 21885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:36.408717 21885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:36.408740 21885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:36.409936 21885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:36.409986 21885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:36.409998 21885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:36.410010 21885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:36.410022 21885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:36.411502 21885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:45623
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:45623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:36.411803 21885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:36.412154 21885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:36.417472 21893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:36.417522 21891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:36.417543 21885 server_base.cc:1034] running on GCE node
W20260812 04:01:36.417547 21890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:36.418048 21885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:36.418354 21885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:36.419543 21885 hybrid_clock.cc:648] HybridClock initialized: now 1786507296419518 us; error 40 us; skew 500 ppm
I20260812 04:01:36.420894 21885 webserver.cc:460] Webserver started at http://127.31.43.254:34943/ using document root <none> and password file <none>
I20260812 04:01:36.421195 21885 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:36.421248 21885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:36.421401 21885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:36.422521 21885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "25695bf16dcc489cba59c8480129287b"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "09121ff4e272ec0caa7be429db430c1c"
server_key_iv: "d3ee7123bc715445f05edc1ad526e58f"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.422940 21885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "25695bf16dcc489cba59c8480129287b"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "09121ff4e272ec0caa7be429db430c1c"
server_key_iv: "d3ee7123bc715445f05edc1ad526e58f"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.424342 21885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:36.425143 21899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.425328 21885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:36.425434 21885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "25695bf16dcc489cba59c8480129287b"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "09121ff4e272ec0caa7be429db430c1c"
server_key_iv: "d3ee7123bc715445f05edc1ad526e58f"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.425520 21885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:36.438998 21885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:36.439345 21885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:36.439517 21885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:36.443770 21885 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:45623
I20260812 04:01:36.443807 21960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:45623 every 8 connection(s)
I20260812 04:01:36.444185 21885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:36.445231 21961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:36.446889 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 21885
I20260812 04:01:36.447089 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:36.447288 31919 external_mini_cluster.cc:1428] Setting key 233835dec858c6268051ce03f1692636
I20260812 04:01:36.447955 21961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b: Bootstrap starting.
I20260812 04:01:36.448593 21961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:36.448977 21961 log.cc:824] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:36.449723 21961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b: No bootstrap required, opened a new log
I20260812 04:01:36.452164 21961 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25695bf16dcc489cba59c8480129287b" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 45623 } }
I20260812 04:01:36.452347 21961 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:36.452387 21961 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25695bf16dcc489cba59c8480129287b, State: Initialized, Role: FOLLOWER
I20260812 04:01:36.452520 21961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25695bf16dcc489cba59c8480129287b" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 45623 } }
I20260812 04:01:36.452615 21961 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:36.452653 21961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:36.452708 21961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:36.453281 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.447873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39276 (local address 127.31.43.254:45623)
0812 04:01:36.448091 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.448101 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.448133 (+ 32us) server_negotiation.cc:407] Connection header received
0812 04:01:36.448598 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.448605 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.448703 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.448853 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:36.449132 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.450117 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.451247 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.451676 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.451776 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.452521 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:36.452525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:36.452526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:36.452752 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:36.452800 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:36.452815 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:36.452887 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.452991 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.453075 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":106,"threads_started":1}
I20260812 04:01:36.453459 21961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25695bf16dcc489cba59c8480129287b" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 45623 } }
I20260812 04:01:36.453549 21961 leader_election.cc:302] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25695bf16dcc489cba59c8480129287b; no voters:
I20260812 04:01:36.453738 21961 leader_election.cc:288] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:36.453841 21968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:36.454001 21961 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:36.454054 21968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [term 1 LEADER]: Becoming Leader. State: Replica: 25695bf16dcc489cba59c8480129287b, State: Running, Role: LEADER
I20260812 04:01:36.454149 21968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25695bf16dcc489cba59c8480129287b" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 45623 } }
I20260812 04:01:36.454480 21969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25695bf16dcc489cba59c8480129287b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25695bf16dcc489cba59c8480129287b" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 45623 } } }
I20260812 04:01:36.454584 21969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:36.454519 21971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25695bf16dcc489cba59c8480129287b. Latest consensus state: current_term: 1 leader_uuid: "25695bf16dcc489cba59c8480129287b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25695bf16dcc489cba59c8480129287b" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 45623 } } }
I20260812 04:01:36.454756 21971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:36.454928 21978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:36.455515 21978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:36.457003 21978 catalog_manager.cc:1347] Generated new cluster ID: 2cbdcaeb036a48bea4382de9ea7fa25d
I20260812 04:01:36.457057 21978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:36.475387 21978 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:36.476058 21978 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:36.479089 21978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b: Generated new TSK 0
I20260812 04:01:36.479302 21978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:36.486209 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.193:0
--local_ip_for_outbound_sockets=127.31.43.193
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:45623
--builtin_ntp_servers=127.31.43.212:33553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 04:01:36.575201 21990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:36.575412 21990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:36.575435 21990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:36.576619 21990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:36.576661 21990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:36.576687 21990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.193
I20260812 04:01:36.578083 21990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.43.193
--webserver_port=0
--tserver_master_addrs=127.31.43.254:45623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.193
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:36.578351 21990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:36.578735 21990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 04:01:36.584343 21990 server_base.cc:1034] running on GCE node
W20260812 04:01:36.584260 21995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:36.584268 21996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:36.584285 21998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:36.584669 21990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:36.584935 21990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:36.586117 21990 hybrid_clock.cc:648] HybridClock initialized: now 1786507296586121 us; error 48 us; skew 500 ppm
I20260812 04:01:36.587448 21990 webserver.cc:460] Webserver started at http://127.31.43.193:39549/ using document root <none> and password file <none>
I20260812 04:01:36.587729 21990 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:36.587803 21990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:36.587951 21990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:36.589038 21990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/data/instance:
uuid: "1602039e68df48258daa8979fb25b418"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "0063504d8032b789ffa8ce0813e695fd"
server_key_iv: "528091209cf07ccae631228fe741dc81"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.589421 21990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance:
uuid: "1602039e68df48258daa8979fb25b418"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "0063504d8032b789ffa8ce0813e695fd"
server_key_iv: "528091209cf07ccae631228fe741dc81"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.590809 21990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:36.591573 22005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.591754 21990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 04:01:36.591857 21990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/wal
uuid: "1602039e68df48258daa8979fb25b418"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "0063504d8032b789ffa8ce0813e695fd"
server_key_iv: "528091209cf07ccae631228fe741dc81"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.591938 21990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:36.606253 21990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:36.606617 21990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:36.606810 21990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:36.607069 21990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:36.607426 21990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:36.607481 21990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.607523 21990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:36.607566 21990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.613905 21990 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.193:40041
I20260812 04:01:36.613963 22132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.193:40041 every 8 connection(s)
I20260812 04:01:36.614318 21990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/data/info.pb
I20260812 04:01:36.619854 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.615199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.193:52419 (local address 127.31.43.254:45623)
0812 04:01:36.615277 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.615279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.615517 (+ 238us) server_negotiation.cc:407] Connection header received
0812 04:01:36.615941 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.615942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.615958 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.616032 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:36.616760 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.617320 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.618322 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.618528 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.618601 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.619514 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:36.619515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:36.619516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:36.619521 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:36.619537 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:36.619542 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:36.619598 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.619765 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.619791 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 04:01:36.620134 22136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.615115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:45623 (local address 127.31.43.193:52419)
0812 04:01:36.615311 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.615354 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.615849 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.616124 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.616127 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.616136 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:36.616659 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.616663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.617417 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.617418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.618218 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.618220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.618623 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.618624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.618799 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.619355 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 04:01:36.619355 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:36.619386 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 04:01:36.619410 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:36.619642 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:36.619650 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:36.619653 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.619729 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.619819 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260812 04:01:36.620736 22134 heartbeater.cc:347] Connected to a master server at 127.31.43.254:45623
I20260812 04:01:36.620829 22134 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:36.621094 22134 heartbeater.cc:511] Master 127.31.43.254:45623 requested a full tablet report, sending...
I20260812 04:01:36.621678 21919 ts_manager.cc:194] Registered new tserver with Master: 1602039e68df48258daa8979fb25b418 (127.31.43.193:40041)
I20260812 04:01:36.622325 21919 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.43.193:52419
I20260812 04:01:36.622954 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 21990
I20260812 04:01:36.623082 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-0/wal/instance
I20260812 04:01:36.623286 31919 external_mini_cluster.cc:1428] Setting key 2a497a67aa189da3d582e42239ccbfd7
I20260812 04:01:36.624339 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.194:0
--local_ip_for_outbound_sockets=127.31.43.194
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:45623
--builtin_ntp_servers=127.31.43.212:33553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 04:01:36.713371 22137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:36.713635 22137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:36.713660 22137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:36.714854 22137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:36.714916 22137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:36.714944 22137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.194
I20260812 04:01:36.716320 22137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.43.194
--webserver_port=0
--tserver_master_addrs=127.31.43.254:45623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.194
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:36.716539 22137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:36.716825 22137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:36.722375 22143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:36.722605 22145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:36.722400 22142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:36.723124 22137 server_base.cc:1034] running on GCE node
I20260812 04:01:36.723382 22137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:36.723711 22137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:36.724891 22137 hybrid_clock.cc:648] HybridClock initialized: now 1786507296724865 us; error 39 us; skew 500 ppm
I20260812 04:01:36.726428 22137 webserver.cc:460] Webserver started at http://127.31.43.194:36941/ using document root <none> and password file <none>
I20260812 04:01:36.726783 22137 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:36.726861 22137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:36.727010 22137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:36.728154 22137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/data/instance:
uuid: "4c74464796014c52ac89a3c6d57e7b49"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "1b33db29aa9528c757bfa493674b844a"
server_key_iv: "e2bc60732dd68c718d23382b8abdb3c4"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.728572 22137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance:
uuid: "4c74464796014c52ac89a3c6d57e7b49"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "1b33db29aa9528c757bfa493674b844a"
server_key_iv: "e2bc60732dd68c718d23382b8abdb3c4"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.729991 22137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:36.730863 22151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.731093 22137 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:36.731192 22137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/wal
uuid: "4c74464796014c52ac89a3c6d57e7b49"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "1b33db29aa9528c757bfa493674b844a"
server_key_iv: "e2bc60732dd68c718d23382b8abdb3c4"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.731283 22137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:36.759011 22137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:36.759402 22137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:36.759579 22137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:36.759845 22137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:36.760231 22137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:36.760290 22137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.760334 22137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:36.760373 22137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.766840 22137 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.194:36739
I20260812 04:01:36.766902 22280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.194:36739 every 8 connection(s)
I20260812 04:01:36.767270 22137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/data/info.pb
I20260812 04:01:36.771214 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 22137
I20260812 04:01:36.771354 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/wal/instance
I20260812 04:01:36.771574 31919 external_mini_cluster.cc:1428] Setting key 3119f10380bf02ed7d958eb94d61ae60
I20260812 04:01:36.772864 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.43.195:0
--local_ip_for_outbound_sockets=127.31.43.195
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:45623
--builtin_ntp_servers=127.31.43.212:33553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 04:01:36.773517 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.768122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.194:54651 (local address 127.31.43.254:45623)
0812 04:01:36.768171 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.768173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.768521 (+ 348us) server_negotiation.cc:407] Connection header received
0812 04:01:36.769087 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.769088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.769101 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.769160 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:36.769927 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.770441 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.771597 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.771924 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.772007 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.773205 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:36.773207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:36.773207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:36.773213 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:36.773230 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:36.773234 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:36.773309 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.773407 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.773435 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 04:01:36.773737 22283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.768159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:45623 (local address 127.31.43.194:54651)
0812 04:01:36.768378 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.768414 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.768975 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.769238 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.769242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.769252 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:36.769783 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.769788 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.770548 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.770549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.771505 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.771508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.772021 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.772022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.772174 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.772969 (+ 795us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 04:01:36.772970 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:36.773030 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 04:01:36.773063 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:36.773310 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:36.773317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:36.773321 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.773389 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.773502 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260812 04:01:36.775362 22281 heartbeater.cc:347] Connected to a master server at 127.31.43.254:45623
I20260812 04:01:36.775487 22281 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:36.775714 22281 heartbeater.cc:511] Master 127.31.43.254:45623 requested a full tablet report, sending...
I20260812 04:01:36.776204 21919 ts_manager.cc:194] Registered new tserver with Master: 4c74464796014c52ac89a3c6d57e7b49 (127.31.43.194:36739)
I20260812 04:01:36.776767 21919 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.43.194:54651
W20260812 04:01:36.870162 22285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:36.870391 22285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:36.870417 22285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:36.871817 22285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:36.871870 22285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:36.871901 22285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.43.195
I20260812 04:01:36.873493 22285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.43.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.43.195
--webserver_port=0
--tserver_master_addrs=127.31.43.254:45623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.43.195
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:36.873826 22285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:36.874204 22285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:36.879987 22292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:36.879987 22294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:36.880072 22291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:36.880086 22285 server_base.cc:1034] running on GCE node
I20260812 04:01:36.880462 22285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:36.880728 22285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:36.881916 22285 hybrid_clock.cc:648] HybridClock initialized: now 1786507296881872 us; error 57 us; skew 500 ppm
I20260812 04:01:36.883411 22285 webserver.cc:460] Webserver started at http://127.31.43.195:40563/ using document root <none> and password file <none>
I20260812 04:01:36.883706 22285 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:36.883759 22285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:36.883939 22285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:36.885068 22285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/data/instance:
uuid: "b1fb53fb6c864ba3bf881a5455cef88c"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "76b0c352eea28207c924f910069a1fad"
server_key_iv: "c29c8eca559167a9370b56f57f2ef4db"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.885442 22285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance:
uuid: "b1fb53fb6c864ba3bf881a5455cef88c"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "76b0c352eea28207c924f910069a1fad"
server_key_iv: "c29c8eca559167a9370b56f57f2ef4db"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.886852 22285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:36.887651 22301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.887867 22285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:36.887948 22285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/wal
uuid: "b1fb53fb6c864ba3bf881a5455cef88c"
format_stamp: "Formatted at 2026-08-12 04:01:36 on dist-test-slave-8tdl"
server_key: "76b0c352eea28207c924f910069a1fad"
server_key_iv: "c29c8eca559167a9370b56f57f2ef4db"
server_key_version: "encryptionkey@0"
I20260812 04:01:36.888044 22285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:36.907876 22285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:36.908185 22285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:36.908303 22285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:36.908486 22285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 04:01:36.908862 22285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 04:01:36.908901 22285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.908944 22285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 04:01:36.908970 22285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:36.915143 22285 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.195:33271
I20260812 04:01:36.915225 22421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.195:33271 every 8 connection(s)
I20260812 04:01:36.915575 22285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/data/info.pb
I20260812 04:01:36.920709 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 22285
I20260812 04:01:36.920857 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/wal/instance
I20260812 04:01:36.921149 31919 external_mini_cluster.cc:1428] Setting key 5c9ae978c488a82de30ed33a2cb03587
I20260812 04:01:36.921154 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.916446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:32879 (local address 127.31.43.254:45623)
0812 04:01:36.916515 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.916517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.916794 (+ 277us) server_negotiation.cc:407] Connection header received
0812 04:01:36.917292 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.917293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.917305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.917360 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:36.918066 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.918555 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.919609 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.919798 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.919856 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.920818 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:36.920820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:36.920820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:36.920826 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:36.920843 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:36.920848 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:36.920905 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.921035 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.921089 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 04:01:36.921536 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.916448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:45623 (local address 127.31.43.195:32879)
0812 04:01:36.916669 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.916705 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.917202 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.917420 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.917423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.917431 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:01:36.917985 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.917990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.918644 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.918645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.919516 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.919519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.919899 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.919901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.920025 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.920645 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 04:01:36.920646 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:01:36.920680 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 04:01:36.920705 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:01:36.920927 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:01:36.920936 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:01:36.920939 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.921033 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.921154 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":114,"threads_started":1}
I20260812 04:01:36.922072 22422 heartbeater.cc:347] Connected to a master server at 127.31.43.254:45623
I20260812 04:01:36.922188 22422 heartbeater.cc:464] Registering TS with master...
I20260812 04:01:36.922379 22422 heartbeater.cc:511] Master 127.31.43.254:45623 requested a full tablet report, sending...
I20260812 04:01:36.922823 21919 ts_manager.cc:194] Registered new tserver with Master: b1fb53fb6c864ba3bf881a5455cef88c (127.31.43.195:33271)
I20260812 04:01:36.923250 21919 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.43.195:32879
I20260812 04:01:36.933180 31919 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 04:01:36.937868 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.934732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39286 (local address 127.31.43.254:45623)
0812 04:01:36.934802 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.934805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.934936 (+ 131us) server_negotiation.cc:407] Connection header received
0812 04:01:36.935006 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.935007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.935016 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.935069 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:36.935257 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.935749 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.936607 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.936799 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.936857 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.937600 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:36.937601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:36.937602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:36.937607 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:36.937622 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:36.937625 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:36.937669 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.937724 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.937748 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 04:01:36.939777 21919 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 04:01:36.941628 21919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 04:01:36.948655 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.944177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.195:33271 (local address 127.0.0.1:39564)
0812 04:01:36.944563 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.944609 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.944692 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.945093 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.945096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.945098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:36.945177 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.945180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.946528 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.946529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.947581 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.947584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.948330 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.948331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.948413 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.948438 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.948504 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.948570 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":22,"thread_start_us":71,"threads_started":1}
I20260812 04:01:36.948668 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.944371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.31.43.195:33271)
0812 04:01:36.944576 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.944580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.944775 (+ 195us) server_negotiation.cc:407] Connection header received
0812 04:01:36.944888 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.944892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.944913 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.944991 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:36.945263 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.946425 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.947680 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.948226 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.948318 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.948421 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.948534 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.948573 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
I20260812 04:01:36.948797 22433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.944186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:40041 (local address 127.0.0.1:59730)
0812 04:01:36.944618 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.944629 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.944867 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.945132 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.945133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.945134 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:36.945177 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.945180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.946163 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.946164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.947441 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.947445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.948529 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.948531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.948607 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.948612 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.948664 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.948710 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260812 04:01:36.948974 22438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.944534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34058 (local address 127.31.43.194:36739)
0812 04:01:36.944724 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.944733 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.944797 (+ 64us) server_negotiation.cc:407] Connection header received
0812 04:01:36.944868 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.944869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.944904 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.945079 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:36.945305 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.946420 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.947745 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.948471 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.948535 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.948617 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.948844 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.948894 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":109,"threads_started":1}
I20260812 04:01:36.949066 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.944269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59730 (local address 127.31.43.193:40041)
0812 04:01:36.944497 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.944503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.944684 (+ 181us) server_negotiation.cc:407] Connection header received
0812 04:01:36.944969 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.944971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.944986 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.945044 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:36.945246 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.946058 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.947542 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.948315 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.948394 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.948489 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.948923 (+ 434us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.948970 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":104,"threads_started":1}
I20260812 04:01:36.949205 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.944401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:36739 (local address 127.0.0.1:34058)
0812 04:01:36.944742 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.944751 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.944806 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.945161 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.945162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.945165 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:36.945224 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.945226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.946515 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.946516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.947670 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.947673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.948650 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.948651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.948707 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.948709 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.948755 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.948788 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":28,"thread_start_us":84,"threads_started":1}
I20260812 04:01:36.950055 22056 tablet_service.cc:1467] Processing CreateTablet for tablet 9740e6ab54e841ca9b62369d6254d945 (DEFAULT_TABLE table=test-table [id=837ee394969f46c289eb93891ed1818d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:36.950088 22350 tablet_service.cc:1467] Processing CreateTablet for tablet 9740e6ab54e841ca9b62369d6254d945 (DEFAULT_TABLE table=test-table [id=837ee394969f46c289eb93891ed1818d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:36.950090 22205 tablet_service.cc:1467] Processing CreateTablet for tablet 9740e6ab54e841ca9b62369d6254d945 (DEFAULT_TABLE table=test-table [id=837ee394969f46c289eb93891ed1818d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 04:01:36.950455 22056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9740e6ab54e841ca9b62369d6254d945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:36.950455 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9740e6ab54e841ca9b62369d6254d945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:36.950455 22205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9740e6ab54e841ca9b62369d6254d945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:36.953424 22439 tablet_bootstrap.cc:492] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418: Bootstrap starting.
I20260812 04:01:36.954105 22440 tablet_bootstrap.cc:492] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c: Bootstrap starting.
I20260812 04:01:36.954248 22441 tablet_bootstrap.cc:492] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49: Bootstrap starting.
I20260812 04:01:36.954419 22439 tablet_bootstrap.cc:654] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:36.954843 22440 tablet_bootstrap.cc:654] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:36.954919 22439 log.cc:824] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:36.954969 22441 tablet_bootstrap.cc:654] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:36.955230 22440 log.cc:824] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:36.955334 22441 log.cc:824] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:36.955933 22439 tablet_bootstrap.cc:492] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418: No bootstrap required, opened a new log
I20260812 04:01:36.956022 22439 ts_tablet_manager.cc:1397] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 04:01:36.956058 22440 tablet_bootstrap.cc:492] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c: No bootstrap required, opened a new log
I20260812 04:01:36.956146 22440 ts_tablet_manager.cc:1397] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 04:01:36.958192 22439 raft_consensus.cc:348] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1602039e68df48258daa8979fb25b418" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40041 } } peers { permanent_uuid: "4c74464796014c52ac89a3c6d57e7b49" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36739 } } peers { permanent_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33271 } }
I20260812 04:01:36.958343 22439 raft_consensus.cc:374] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:36.958366 22439 raft_consensus.cc:729] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1602039e68df48258daa8979fb25b418, State: Initialized, Role: FOLLOWER
I20260812 04:01:36.958262 22440 raft_consensus.cc:348] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1602039e68df48258daa8979fb25b418" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40041 } } peers { permanent_uuid: "4c74464796014c52ac89a3c6d57e7b49" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36739 } } peers { permanent_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33271 } }
I20260812 04:01:36.958447 22440 raft_consensus.cc:374] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:36.958492 22440 raft_consensus.cc:729] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1fb53fb6c864ba3bf881a5455cef88c, State: Initialized, Role: FOLLOWER
I20260812 04:01:36.958500 22439 consensus_queue.cc:260] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1602039e68df48258daa8979fb25b418" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40041 } } peers { permanent_uuid: "4c74464796014c52ac89a3c6d57e7b49" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36739 } } peers { permanent_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33271 } }
I20260812 04:01:36.958608 22440 consensus_queue.cc:260] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1602039e68df48258daa8979fb25b418" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40041 } } peers { permanent_uuid: "4c74464796014c52ac89a3c6d57e7b49" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36739 } } peers { permanent_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33271 } }
I20260812 04:01:36.958856 22134 heartbeater.cc:503] Master 127.31.43.254:45623 was elected leader, sending a full tablet report...
I20260812 04:01:36.958910 22440 ts_tablet_manager.cc:1428] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 04:01:36.959038 22422 heartbeater.cc:503] Master 127.31.43.254:45623 was elected leader, sending a full tablet report...
I20260812 04:01:36.958845 22439 ts_tablet_manager.cc:1428] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 04:01:36.959841 22441 tablet_bootstrap.cc:492] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49: No bootstrap required, opened a new log
I20260812 04:01:36.959928 22441 ts_tablet_manager.cc:1397] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260812 04:01:36.962225 22441 raft_consensus.cc:348] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1602039e68df48258daa8979fb25b418" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40041 } } peers { permanent_uuid: "4c74464796014c52ac89a3c6d57e7b49" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36739 } } peers { permanent_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33271 } }
I20260812 04:01:36.962375 22441 raft_consensus.cc:374] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:36.962397 22441 raft_consensus.cc:729] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c74464796014c52ac89a3c6d57e7b49, State: Initialized, Role: FOLLOWER
I20260812 04:01:36.962520 22441 consensus_queue.cc:260] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1602039e68df48258daa8979fb25b418" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40041 } } peers { permanent_uuid: "4c74464796014c52ac89a3c6d57e7b49" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36739 } } peers { permanent_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33271 } }
I20260812 04:01:36.962842 22441 ts_tablet_manager.cc:1428] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 04:01:36.962888 22281 heartbeater.cc:503] Master 127.31.43.254:45623 was elected leader, sending a full tablet report...
I20260812 04:01:36.966913 22447 raft_consensus.cc:482] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 04:01:36.967094 22447 raft_consensus.cc:504] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1602039e68df48258daa8979fb25b418" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40041 } } peers { permanent_uuid: "4c74464796014c52ac89a3c6d57e7b49" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36739 } } peers { permanent_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33271 } }
I20260812 04:01:36.967533 22447 leader_election.cc:288] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1602039e68df48258daa8979fb25b418 (127.31.43.193:40041), 4c74464796014c52ac89a3c6d57e7b49 (127.31.43.194:36739)
I20260812 04:01:36.970544 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.967794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:49107 (local address 127.31.43.193:40041)
0812 04:01:36.967837 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.967839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.967849 (+ 10us) server_negotiation.cc:407] Connection header received
0812 04:01:36.967913 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.967914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.967925 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.968021 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:36.968210 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.968795 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.969814 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.970196 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.970253 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.970260 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.970426 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.970476 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 04:01:36.970544 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.967731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:40041 (local address 127.31.43.195:49107)
0812 04:01:36.967792 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.967805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.967849 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.968035 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.968036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.968038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:36.968137 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.968139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.968878 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.968879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.969721 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.969724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.970279 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.970280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.970353 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.970356 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.970411 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.970462 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":31}
I20260812 04:01:36.970605 22438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.968103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.43.195:55707 (local address 127.31.43.194:36739)
0812 04:01:36.968172 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.968174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.968184 (+ 10us) server_negotiation.cc:407] Connection header received
0812 04:01:36.968209 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.968210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.968222 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.968273 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 04:01:36.968419 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.968886 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.969844 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.970209 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.970302 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.970310 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.970489 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.970530 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 04:01:36.970641 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.967860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.194:36739 (local address 127.31.43.195:55707)
0812 04:01:36.968052 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:36.968063 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:01:36.968113 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:01:36.968290 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:01:36.968291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:01:36.968293 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 04:01:36.968356 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.968358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.968968 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.968969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.969732 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:01:36.969734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.970326 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.970327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:01:36.970406 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.970411 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 04:01:36.970475 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 04:01:36.970512 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":109,"threads_started":1}
I20260812 04:01:36.970932 22077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9740e6ab54e841ca9b62369d6254d945" candidate_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1602039e68df48258daa8979fb25b418" is_pre_election: true
I20260812 04:01:36.970939 22229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9740e6ab54e841ca9b62369d6254d945" candidate_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c74464796014c52ac89a3c6d57e7b49" is_pre_election: true
I20260812 04:01:36.971107 22229 raft_consensus.cc:2454] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1fb53fb6c864ba3bf881a5455cef88c in term 0.
I20260812 04:01:36.971107 22077 raft_consensus.cc:2454] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1fb53fb6c864ba3bf881a5455cef88c in term 0.
I20260812 04:01:36.971340 22302 leader_election.cc:302] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1602039e68df48258daa8979fb25b418, b1fb53fb6c864ba3bf881a5455cef88c; no voters:
I20260812 04:01:36.971482 22447 raft_consensus.cc:2781] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 04:01:36.971561 22447 raft_consensus.cc:482] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 04:01:36.971619 22447 raft_consensus.cc:3037] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:36.972379 22447 raft_consensus.cc:504] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1602039e68df48258daa8979fb25b418" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40041 } } peers { permanent_uuid: "4c74464796014c52ac89a3c6d57e7b49" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36739 } } peers { permanent_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33271 } }
I20260812 04:01:36.972530 22447 leader_election.cc:288] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [CANDIDATE]: Term 1 election: Requested vote from peers 1602039e68df48258daa8979fb25b418 (127.31.43.193:40041), 4c74464796014c52ac89a3c6d57e7b49 (127.31.43.194:36739)
I20260812 04:01:36.972741 22077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9740e6ab54e841ca9b62369d6254d945" candidate_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1602039e68df48258daa8979fb25b418"
I20260812 04:01:36.972740 22229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9740e6ab54e841ca9b62369d6254d945" candidate_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c74464796014c52ac89a3c6d57e7b49"
I20260812 04:01:36.972833 22077 raft_consensus.cc:3037] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:36.972833 22229 raft_consensus.cc:3037] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:36.973803 22229 raft_consensus.cc:2454] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1fb53fb6c864ba3bf881a5455cef88c in term 1.
I20260812 04:01:36.973804 22077 raft_consensus.cc:2454] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1fb53fb6c864ba3bf881a5455cef88c in term 1.
I20260812 04:01:36.973970 22302 leader_election.cc:302] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1602039e68df48258daa8979fb25b418, b1fb53fb6c864ba3bf881a5455cef88c; no voters:
I20260812 04:01:36.974130 22447 raft_consensus.cc:2781] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:36.974318 22447 raft_consensus.cc:686] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 1 LEADER]: Becoming Leader. State: Replica: b1fb53fb6c864ba3bf881a5455cef88c, State: Running, Role: LEADER
I20260812 04:01:36.974428 22447 consensus_queue.cc:237] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1602039e68df48258daa8979fb25b418" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40041 } } peers { permanent_uuid: "4c74464796014c52ac89a3c6d57e7b49" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36739 } } peers { permanent_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33271 } }
I20260812 04:01:36.975414 21918 catalog_manager.cc:5495] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c reported cstate change: term changed from 0 to 1, leader changed from <none> to b1fb53fb6c864ba3bf881a5455cef88c (127.31.43.195). New cstate: current_term: 1 leader_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1602039e68df48258daa8979fb25b418" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c74464796014c52ac89a3c6d57e7b49" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1fb53fb6c864ba3bf881a5455cef88c" member_type: VOTER last_known_addr { host: "127.31.43.195" port: 33271 } health_report { overall_health: HEALTHY } } }
I20260812 04:01:36.986457 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:36.983941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39580 (local address 127.31.43.195:33271)
0812 04:01:36.984011 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:36.984014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:36.984023 (+ 9us) server_negotiation.cc:407] Connection header received
0812 04:01:36.984069 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:36.984069 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:36.984087 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:36.984169 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:01:36.984290 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.984803 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.985599 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:36.985797 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:36.985878 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:36.986064 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:01:36.986181 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:01:36.986285 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:36.986304 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 04:01:36.986341 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 04:01:36.988441 22077 raft_consensus.cc:1261] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418 [term 1 FOLLOWER]: Refusing update from remote peer b1fb53fb6c864ba3bf881a5455cef88c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:01:36.988523 22229 raft_consensus.cc:1261] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49 [term 1 FOLLOWER]: Refusing update from remote peer b1fb53fb6c864ba3bf881a5455cef88c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:01:36.988826 22450 consensus_queue.cc:1035] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1602039e68df48258daa8979fb25b418" member_type: VOTER last_known_addr { host: "127.31.43.193" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:01:36.989020 22447 consensus_queue.cc:1035] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c74464796014c52ac89a3c6d57e7b49" member_type: VOTER last_known_addr { host: "127.31.43.194" port: 36739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:01:36.992079 22455 mvcc.cc:205] Tried to move back new op lower bound from 7317533888461574144 to 7317533888408244224. Current Snapshot: MvccSnapshot[applied={T|T < 7317533888461574144}]
I20260812 04:01:36.992914 22454 mvcc.cc:205] Tried to move back new op lower bound from 7317533888461574144 to 7317533888408244224. Current Snapshot: MvccSnapshot[applied={T|T < 7317533888461574144}]
I20260812 04:01:36.994602 21918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:01:36.994750 21918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 04:01:36.995579 21918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25695bf16dcc489cba59c8480129287b: Sending DeleteTablet for 3 replicas of tablet 9740e6ab54e841ca9b62369d6254d945
I20260812 04:01:36.995908 22056 tablet_service.cc:1514] Processing DeleteTablet for tablet 9740e6ab54e841ca9b62369d6254d945 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:36 UTC) from {username='slave'} at 127.0.0.1:59730
I20260812 04:01:36.995976 22205 tablet_service.cc:1514] Processing DeleteTablet for tablet 9740e6ab54e841ca9b62369d6254d945 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:36 UTC) from {username='slave'} at 127.0.0.1:34058
I20260812 04:01:36.996246 22464 tablet_replica.cc:330] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49: stopping tablet replica
I20260812 04:01:36.996237 22463 tablet_replica.cc:330] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418: stopping tablet replica
I20260812 04:01:36.996357 22464 raft_consensus.cc:2229] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:01:36.996358 22463 raft_consensus.cc:2229] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 04:01:36.996353 22350 tablet_service.cc:1514] Processing DeleteTablet for tablet 9740e6ab54e841ca9b62369d6254d945 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 04:01:36 UTC) from {username='slave'} at 127.0.0.1:39564
I20260812 04:01:36.996470 22464 raft_consensus.cc:2258] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:36.996467 22463 raft_consensus.cc:2258] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:36.996866 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 21990
I20260812 04:01:36.996939 22463 ts_tablet_manager.cc:1905] T 9740e6ab54e841ca9b62369d6254d945 P 1602039e68df48258daa8979fb25b418: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:01:37.001966 22465 tablet_replica.cc:330] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c: stopping tablet replica
I20260812 04:01:37.002125 22465 raft_consensus.cc:2229] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 1 LEADER]: Raft consensus shutting down.
I20260812 04:01:37.002285 22465 raft_consensus.cc:2258] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 04:01:37.002820 22465 ts_tablet_manager.cc:1905] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:01:37.003014 22464 ts_tablet_manager.cc:1905] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 04:01:37.004714 22465 ts_tablet_manager.cc:1918] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:01:37.004796 22465 log.cc:1196] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-2/wal/wals/9740e6ab54e841ca9b62369d6254d945
I20260812 04:01:37.004964 22464 ts_tablet_manager.cc:1918] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 04:01:37.005074 22465 ts_tablet_manager.cc:1939] T 9740e6ab54e841ca9b62369d6254d945 P b1fb53fb6c864ba3bf881a5455cef88c: Deleting consensus metadata
I20260812 04:01:37.005175 22464 log.cc:1196] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49: Deleting WAL directory at /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786507041847525-31919-0/minicluster-data/ts-1/wal/wals/9740e6ab54e841ca9b62369d6254d945
I20260812 04:01:37.005524 22464 ts_tablet_manager.cc:1939] T 9740e6ab54e841ca9b62369d6254d945 P 4c74464796014c52ac89a3c6d57e7b49: Deleting consensus metadata
I20260812 04:01:37.005589 21903 catalog_manager.cc:4841] TS b1fb53fb6c864ba3bf881a5455cef88c (127.31.43.195:33271): tablet 9740e6ab54e841ca9b62369d6254d945 (table test-table [id=837ee394969f46c289eb93891ed1818d]) successfully deleted
I20260812 04:01:37.006059 21901 catalog_manager.cc:4841] TS 4c74464796014c52ac89a3c6d57e7b49 (127.31.43.194:36739): tablet 9740e6ab54e841ca9b62369d6254d945 (table test-table [id=837ee394969f46c289eb93891ed1818d]) successfully deleted
W20260812 04:01:37.007370 21901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.43.193:40041 (error 108)
I20260812 04:01:37.007649 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 22137
I20260812 04:01:37.016129 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 22285
I20260812 04:01:37.019075 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:37.018888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.193:40041 (local address 127.0.0.1:59746)
0812 04:01:37.018960 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 04:01:37.019023 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.193:40041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 04:01:37.019229 21901 catalog_manager.cc:4584] TS 1602039e68df48258daa8979fb25b418 (127.31.43.193:40041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9740e6ab54e841ca9b62369d6254d945: Network error: Client connection negotiation failed: client connection to 127.31.43.193:40041: connect: Connection refused (error 111)
I20260812 04:01:37.024082 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 21885
2026-08-12T04:01:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (733 ms)
[----------] 6 tests from EncryptionPolicyTest (8988 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:37 dist-test-slave-8tdl krb5kdc[22469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:37 dist-test-slave-8tdl krb5kdc[22469](info): set up 2 sockets
Aug 12 04:01:37 dist-test-slave-8tdl krb5kdc[22469](info): commencing operation
krb5kdc: starting...
W20260812 04:01:38.152806 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:38 dist-test-slave-8tdl krb5kdc[22469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 04:01:38.295271 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34661
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:38.389283 22484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:38.389596 22484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:38.389648 22484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:38.391072 22484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:38.391163 22484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:38.391189 22484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:38.391222 22484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:38.391248 22484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:38.393138 22484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34661
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:38.393498 22484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:38.393783 22484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:38.399573 22490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:38.399591 22493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:38.399605 22491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:38.399847 22484 server_base.cc:1034] running on GCE node
I20260812 04:01:38.400126 22484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:38.400465 22484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:38.401661 22484 hybrid_clock.cc:648] HybridClock initialized: now 1786507298401639 us; error 41 us; skew 500 ppm
Aug 12 04:01:38 dist-test-slave-8tdl krb5kdc[22469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507298, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:38.404959 22484 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 04:01:38.405473 22484 webserver.cc:460] Webserver started at http://127.0.0.1:39029/ using document root <none> and password file <none>
I20260812 04:01:38.405730 22484 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:38.405804 22484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:38.405972 22484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:38.407225 22484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "3a07275ed1b449cf9de7488757513d38"
format_stamp: "Formatted at 2026-08-12 04:01:38 on dist-test-slave-8tdl"
server_key: "9236488f0fb5651d2f8b308144f8bdb1"
server_key_iv: "03194fbcefec87972be641aef0b745c4"
server_key_version: "encryptionkey@0"
I20260812 04:01:38.407613 22484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "3a07275ed1b449cf9de7488757513d38"
format_stamp: "Formatted at 2026-08-12 04:01:38 on dist-test-slave-8tdl"
server_key: "9236488f0fb5651d2f8b308144f8bdb1"
server_key_iv: "03194fbcefec87972be641aef0b745c4"
server_key_version: "encryptionkey@0"
I20260812 04:01:38.409132 22484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:01:38.410107 22501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:38.410460 22484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:38.410715 22484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "3a07275ed1b449cf9de7488757513d38"
format_stamp: "Formatted at 2026-08-12 04:01:38 on dist-test-slave-8tdl"
server_key: "9236488f0fb5651d2f8b308144f8bdb1"
server_key_iv: "03194fbcefec87972be641aef0b745c4"
server_key_version: "encryptionkey@0"
I20260812 04:01:38.410809 22484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:38.423205 22484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:38.425707 22484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:38.425918 22484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:38.430505 22484 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34661
I20260812 04:01:38.430604 22559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34661 every 8 connection(s)
I20260812 04:01:38.431003 22484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:38.431986 22560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:38.432499 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 22484
I20260812 04:01:38.432655 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:38.432891 31919 external_mini_cluster.cc:1428] Setting key b81c62a5259f4f3705a11aab6ed2979b
I20260812 04:01:38.435617 22560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38: Bootstrap starting.
I20260812 04:01:38.436368 22560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:38.436682 22560 log.cc:824] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:38.437423 22560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38: No bootstrap required, opened a new log
I20260812 04:01:38.439627 22560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a07275ed1b449cf9de7488757513d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34661 } }
I20260812 04:01:38.439827 22560 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:38.439877 22560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a07275ed1b449cf9de7488757513d38, State: Initialized, Role: FOLLOWER
I20260812 04:01:38.440050 22560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a07275ed1b449cf9de7488757513d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34661 } }
I20260812 04:01:38.440152 22560 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:38.440191 22560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:38.440240 22560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 04:01:38 dist-test-slave-8tdl krb5kdc[22469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 04:01:38.441092 22560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a07275ed1b449cf9de7488757513d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34661 } }
I20260812 04:01:38.441241 22560 leader_election.cc:302] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a07275ed1b449cf9de7488757513d38; no voters:
I20260812 04:01:38.441463 22560 leader_election.cc:288] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:38.441610 22565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:38.441767 22560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:38.442245 22565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [term 1 LEADER]: Becoming Leader. State: Replica: 3a07275ed1b449cf9de7488757513d38, State: Running, Role: LEADER
I20260812 04:01:38.443553 22565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a07275ed1b449cf9de7488757513d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34661 } }
I20260812 04:01:38.444275 22567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a07275ed1b449cf9de7488757513d38. Latest consensus state: current_term: 1 leader_uuid: "3a07275ed1b449cf9de7488757513d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a07275ed1b449cf9de7488757513d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34661 } } }
I20260812 04:01:38.444289 22568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a07275ed1b449cf9de7488757513d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a07275ed1b449cf9de7488757513d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34661 } } }
W20260812 04:01:38.444868 22582 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 04:01:38.446234 22567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [sys.catalog]: This master's current role is: LEADER
W20260812 04:01:38.446271 22582 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:01:38.446250 22568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:38.445935 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:38.433527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39159 (local address 127.0.0.1:34661)
0812 04:01:38.433755 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:38.433765 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:38.433801 (+ 36us) server_negotiation.cc:407] Connection header received
0812 04:01:38.434347 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:38.434356 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:38.434465 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:38.434627 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:38.435392 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:38.436753 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:38.437595 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:38.438087 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:38.438225 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:38.441042 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:38.441047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:38.441047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:38.441072 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:38.443339 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:38.444023 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:38.444027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:38.444027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:38.444169 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:38.444497 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:38.444499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:38.444499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:38.444848 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:38.445011 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:38.445603 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 04:01:38.445719 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":109,"threads_started":1}
I20260812 04:01:38.446822 22585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:38.447184 22585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:38.447635 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:38.452024 22585 catalog_manager.cc:1347] Generated new cluster ID: 206b50496b5a4993aed1c0b8bdcaa338
I20260812 04:01:38.452088 22585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:38.456846 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:38.448991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53573 (local address 127.0.0.1:34661)
0812 04:01:38.449032 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:38.449033 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:38.449176 (+ 143us) server_negotiation.cc:407] Connection header received
0812 04:01:38.449203 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:38.449204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:38.449215 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:38.449283 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:38.449793 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:38.450937 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:38.451914 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:38.452233 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:38.452308 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:38.453870 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:38.453871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:38.453871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:38.453878 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:38.455561 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:38.456005 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:38.456006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:38.456007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:38.456063 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:38.456302 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:38.456303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:38.456303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:38.456423 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:38.456524 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:38.456692 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 04:01:38.456753 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 04:01:38.461750 22585 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:38.462321 22585 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:38.464696 22585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a07275ed1b449cf9de7488757513d38: Generated new TSK 0
I20260812 04:01:38.464898 22585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:38.470961 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 22484
W20260812 04:01:38.481324 22467 connection.cc:535] client connection to 127.0.0.1:34661 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34661): Connection reset by peer (error 104)
2026-08-12T04:01:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:38 dist-test-slave-8tdl krb5kdc[22595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:38 dist-test-slave-8tdl krb5kdc[22595](info): set up 2 sockets
Aug 12 04:01:38 dist-test-slave-8tdl krb5kdc[22595](info): commencing operation
krb5kdc: starting...
W20260812 04:01:39.596285 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:39 dist-test-slave-8tdl krb5kdc[22595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 04:01:39.731009 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43663
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:39.819355 22613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:39.819625 22613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:39.819662 22613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:39.820865 22613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:39.820941 22613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:39.820967 22613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:39.820988 22613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:39.821020 22613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:39.822625 22613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43663
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:39.822916 22613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:39.823107 22613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:39.828305 22619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:39.828313 22620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:39.828313 22622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:39.828758 22613 server_base.cc:1034] running on GCE node
I20260812 04:01:39.829154 22613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:39.829519 22613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:39.830756 22613 hybrid_clock.cc:648] HybridClock initialized: now 1786507299830724 us; error 41 us; skew 500 ppm
Aug 12 04:01:39 dist-test-slave-8tdl krb5kdc[22595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507299, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:39.833607 22613 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 04:01:39.834075 22613 webserver.cc:460] Webserver started at http://127.0.0.1:42101/ using document root <none> and password file <none>
I20260812 04:01:39.834339 22613 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:39.834389 22613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:39.834534 22613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:39.835744 22613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "ec5e6b704d674ed59ace26e2a1ca464f"
format_stamp: "Formatted at 2026-08-12 04:01:39 on dist-test-slave-8tdl"
server_key: "12e15dd6482cd3f25b5f55f7eb5d96ec"
server_key_iv: "0711973c2dda4bcf5283ee105c9aaec4"
server_key_version: "encryptionkey@0"
I20260812 04:01:39.836161 22613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "ec5e6b704d674ed59ace26e2a1ca464f"
format_stamp: "Formatted at 2026-08-12 04:01:39 on dist-test-slave-8tdl"
server_key: "12e15dd6482cd3f25b5f55f7eb5d96ec"
server_key_iv: "0711973c2dda4bcf5283ee105c9aaec4"
server_key_version: "encryptionkey@0"
I20260812 04:01:39.837680 22613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 04:01:39.838649 22629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:39.838920 22613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:39.839030 22613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "ec5e6b704d674ed59ace26e2a1ca464f"
format_stamp: "Formatted at 2026-08-12 04:01:39 on dist-test-slave-8tdl"
server_key: "12e15dd6482cd3f25b5f55f7eb5d96ec"
server_key_iv: "0711973c2dda4bcf5283ee105c9aaec4"
server_key_version: "encryptionkey@0"
I20260812 04:01:39.839087 22613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:39.857956 22613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:39.860646 22613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:39.860781 22613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:39.865763 22613 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43663
I20260812 04:01:39.865839 22686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43663 every 8 connection(s)
I20260812 04:01:39.866281 22613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:39.867343 22687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:39.867849 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 22613
I20260812 04:01:39.868008 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:39.868270 31919 external_mini_cluster.cc:1428] Setting key 38cb77fc6206f9d871757fddc177bcc6
I20260812 04:01:39.871043 22687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f: Bootstrap starting.
I20260812 04:01:39.871869 22687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:39.872262 22687 log.cc:824] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:39.873225 22687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f: No bootstrap required, opened a new log
I20260812 04:01:39.875633 22690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:39.868978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58241 (local address 127.0.0.1:43663)
0812 04:01:39.869205 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:39.869216 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:39.869256 (+ 40us) server_negotiation.cc:407] Connection header received
0812 04:01:39.869844 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:39.869853 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:39.869919 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:39.870098 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:39.870353 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:39.871914 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:39.873069 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:39.873666 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:39.873828 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:39.874761 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:39.874766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:39.874766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:39.875001 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:39.875038 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:39.875051 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:39.875125 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:39.875341 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 04:01:39.875381 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":94,"threads_started":1}
I20260812 04:01:39.876044 22687 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5e6b704d674ed59ace26e2a1ca464f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43663 } }
I20260812 04:01:39.876235 22687 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:39.876291 22687 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5e6b704d674ed59ace26e2a1ca464f, State: Initialized, Role: FOLLOWER
I20260812 04:01:39.876426 22687 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5e6b704d674ed59ace26e2a1ca464f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43663 } }
I20260812 04:01:39.876533 22687 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:39.876578 22687 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:39.876632 22687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:39.876742 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:39.877509 22687 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5e6b704d674ed59ace26e2a1ca464f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43663 } }
I20260812 04:01:39.877643 22687 leader_election.cc:302] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec5e6b704d674ed59ace26e2a1ca464f; no voters:
I20260812 04:01:39.877872 22687 leader_election.cc:288] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:39.878049 22692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:39.878180 22687 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:39.878258 22692 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [term 1 LEADER]: Becoming Leader. State: Replica: ec5e6b704d674ed59ace26e2a1ca464f, State: Running, Role: LEADER
I20260812 04:01:39.878369 22692 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5e6b704d674ed59ace26e2a1ca464f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43663 } }
I20260812 04:01:39.878875 22695 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec5e6b704d674ed59ace26e2a1ca464f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5e6b704d674ed59ace26e2a1ca464f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43663 } } }
I20260812 04:01:39.878993 22695 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:39.878830 22692 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec5e6b704d674ed59ace26e2a1ca464f. Latest consensus state: current_term: 1 leader_uuid: "ec5e6b704d674ed59ace26e2a1ca464f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5e6b704d674ed59ace26e2a1ca464f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43663 } } }
I20260812 04:01:39.879230 22692 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:39.879364 22706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:39.879954 22706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:39.881495 22717 mvcc.cc:205] Tried to move back new op lower bound from 7317533900312694784 to 7317533900302200832. Current Snapshot: MvccSnapshot[applied={T|T < 7317533900312694784}]
I20260812 04:01:39.882540 22706 catalog_manager.cc:1347] Generated new cluster ID: 7c46cd4884914f739178aa85d99edbfc
I20260812 04:01:39.882611 22706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:39.883097 22690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:39.878484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54389 (local address 127.0.0.1:43663)
0812 04:01:39.878527 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:39.878529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:39.878947 (+ 418us) server_negotiation.cc:407] Connection header received
0812 04:01:39.879145 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:39.879146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:39.879157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:39.879218 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:39.879461 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:39.880229 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:39.881395 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:39.881702 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:39.881748 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:39.882704 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:39.882705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:39.882705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:39.882712 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:39.882730 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:39.882735 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:39.882804 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:39.882996 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 04:01:39.883007 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 04:01:39.901371 22706 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:39.902047 22706 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:39.911194 22706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec5e6b704d674ed59ace26e2a1ca464f: Generated new TSK 0
I20260812 04:01:39.911473 22706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:39.919111 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 22613
W20260812 04:01:39.926836 22593 connection.cc:535] client connection to 127.0.0.1:43663 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43663): Connection reset by peer (error 104)
2026-08-12T04:01:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1445 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:39 dist-test-slave-8tdl krb5kdc[22721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:39 dist-test-slave-8tdl krb5kdc[22721](info): set up 2 sockets
Aug 12 04:01:39 dist-test-slave-8tdl krb5kdc[22721](info): commencing operation
krb5kdc: starting...
W20260812 04:01:41.043118 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:41 dist-test-slave-8tdl krb5kdc[22721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 04:01:41.182808 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38867
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:41.270356 22736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:41.270605 22736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:41.270632 22736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:41.271941 22736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:41.272007 22736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:41.272022 22736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:41.272035 22736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:41.272050 22736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:41.273648 22736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38867
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:41.273888 22736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:41.274096 22736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:41.279999 22742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:41.279999 22743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:41.280162 22745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:41.280224 22736 server_base.cc:1034] running on GCE node
I20260812 04:01:41.280572 22736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:41.280911 22736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:41.282101 22736 hybrid_clock.cc:648] HybridClock initialized: now 1786507301282052 us; error 38 us; skew 500 ppm
Aug 12 04:01:41 dist-test-slave-8tdl krb5kdc[22721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507301, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:41.291356 22736 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 04:01:41.291961 22736 webserver.cc:460] Webserver started at http://127.0.0.1:39751/ using document root <none> and password file <none>
I20260812 04:01:41.292310 22736 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:41.292430 22736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:41.292640 22736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:41.293820 22736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "c84ad886708c47e6980979cf52d00c7c"
format_stamp: "Formatted at 2026-08-12 04:01:41 on dist-test-slave-8tdl"
server_key: "5c85819d62f33c0c55d4bb68e8f80463"
server_key_iv: "c81537ba1d78cf60fa459f26c6629576"
server_key_version: "encryptionkey@0"
I20260812 04:01:41.294243 22736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "c84ad886708c47e6980979cf52d00c7c"
format_stamp: "Formatted at 2026-08-12 04:01:41 on dist-test-slave-8tdl"
server_key: "5c85819d62f33c0c55d4bb68e8f80463"
server_key_iv: "c81537ba1d78cf60fa459f26c6629576"
server_key_version: "encryptionkey@0"
I20260812 04:01:41.295835 22736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 04:01:41.296695 22753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:41.296916 22736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:41.297036 22736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "c84ad886708c47e6980979cf52d00c7c"
format_stamp: "Formatted at 2026-08-12 04:01:41 on dist-test-slave-8tdl"
server_key: "5c85819d62f33c0c55d4bb68e8f80463"
server_key_iv: "c81537ba1d78cf60fa459f26c6629576"
server_key_version: "encryptionkey@0"
I20260812 04:01:41.297138 22736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:41.300895 22736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:41.303486 22736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:41.303696 22736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:41.308548 22736 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38867
I20260812 04:01:41.308597 22809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38867 every 8 connection(s)
I20260812 04:01:41.309047 22736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:41.310182 22810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:41.312906 22810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c: Bootstrap starting.
I20260812 04:01:41.313627 22810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:41.313946 22810 log.cc:824] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:41.314886 22810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c: No bootstrap required, opened a new log
I20260812 04:01:41.316669 22810 raft_consensus.cc:348] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84ad886708c47e6980979cf52d00c7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38867 } }
I20260812 04:01:41.316838 22810 raft_consensus.cc:374] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:41.316864 22810 raft_consensus.cc:729] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84ad886708c47e6980979cf52d00c7c, State: Initialized, Role: FOLLOWER
I20260812 04:01:41.317024 22810 consensus_queue.cc:260] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84ad886708c47e6980979cf52d00c7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38867 } }
I20260812 04:01:41.317097 22810 raft_consensus.cc:388] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:41.317121 22810 raft_consensus.cc:482] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:41.317148 22810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:41.317757 22810 raft_consensus.cc:504] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84ad886708c47e6980979cf52d00c7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38867 } }
I20260812 04:01:41.317852 22810 leader_election.cc:302] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c84ad886708c47e6980979cf52d00c7c; no voters:
I20260812 04:01:41.318089 22810 leader_election.cc:288] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:41.318154 22813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:41.318418 22810 sys_catalog.cc:564] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:41.318643 22813 raft_consensus.cc:686] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [term 1 LEADER]: Becoming Leader. State: Replica: c84ad886708c47e6980979cf52d00c7c, State: Running, Role: LEADER
I20260812 04:01:41.318810 22813 consensus_queue.cc:237] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84ad886708c47e6980979cf52d00c7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38867 } }
I20260812 04:01:41.319248 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 22736
I20260812 04:01:41.319381 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:41.319653 31919 external_mini_cluster.cc:1428] Setting key 76afabb748d916267ffe9142c2d22e49
I20260812 04:01:41.320420 22816 sys_catalog.cc:455] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c84ad886708c47e6980979cf52d00c7c. Latest consensus state: current_term: 1 leader_uuid: "c84ad886708c47e6980979cf52d00c7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84ad886708c47e6980979cf52d00c7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38867 } } }
I20260812 04:01:41.320544 22816 sys_catalog.cc:458] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:41.320773 22814 sys_catalog.cc:455] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c84ad886708c47e6980979cf52d00c7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84ad886708c47e6980979cf52d00c7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38867 } } }
I20260812 04:01:41.320822 22832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:41.320855 22814 sys_catalog.cc:458] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:41.321399 22832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:41.322149 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:41.320181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43307 (local address 127.0.0.1:38867)
0812 04:01:41.320356 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:41.320365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:41.320472 (+ 107us) server_negotiation.cc:407] Connection header received
0812 04:01:41.320664 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:41.320673 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:41.320705 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:41.321171 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:41.321681 (+ 510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:41.321685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:41.321685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:41.321691 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:41.321747 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:41.321761 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:41.321860 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:41.321927 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 04:01:41.321981 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260812 04:01:41.323248 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:41.323457 22832 catalog_manager.cc:1347] Generated new cluster ID: ca0837c4880341ba829328965f933985
I20260812 04:01:41.323516 22832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:41.332351 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:41.324703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42225 (local address 127.0.0.1:38867)
0812 04:01:41.324740 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:41.324742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:41.330957 (+ 6215us) server_negotiation.cc:407] Connection header received
0812 04:01:41.331089 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:41.331090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:41.331107 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:41.331199 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:41.331991 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:41.331992 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:41.331993 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:41.331998 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:41.332017 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:41.332021 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:41.332161 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:41.332232 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 04:01:41.332251 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 04:01:41.346315 22832 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:41.346577 22832 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:41.351673 22832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c84ad886708c47e6980979cf52d00c7c: Generated new TSK 0
I20260812 04:01:41.351922 22832 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:41.355454 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 22736
2026-08-12T04:01:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:41 dist-test-slave-8tdl krb5kdc[22844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:41 dist-test-slave-8tdl krb5kdc[22844](info): set up 2 sockets
Aug 12 04:01:41 dist-test-slave-8tdl krb5kdc[22844](info): commencing operation
krb5kdc: starting...
W20260812 04:01:42.507138 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:42 dist-test-slave-8tdl krb5kdc[22844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 04:01:42.646869 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:42.732969 22859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:42.733234 22859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:42.733275 22859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:42.734512 22859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:42.734588 22859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:42.734613 22859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:42.734637 22859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:42.734665 22859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:42.736330 22859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:42.736651 22859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:42.736908 22859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:42.742210 22865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:42.742223 22868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:42.742230 22866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:42.742641 22859 server_base.cc:1034] running on GCE node
I20260812 04:01:42.742992 22859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:42.743327 22859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:42.744483 22859 hybrid_clock.cc:648] HybridClock initialized: now 1786507302744449 us; error 51 us; skew 500 ppm
Aug 12 04:01:42 dist-test-slave-8tdl krb5kdc[22844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507302, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:42.747509 22859 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 04:01:42.748056 22859 webserver.cc:460] Webserver started at http://127.0.0.1:45927/ using document root <none> and password file <none>
I20260812 04:01:42.748298 22859 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:42.748343 22859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:42.748446 22859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:42.749617 22859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "699da80645f149308aa0b17c0d4076a8"
format_stamp: "Formatted at 2026-08-12 04:01:42 on dist-test-slave-8tdl"
server_key: "7cb6481983d6cab5bfdc26c4492a5ba7"
server_key_iv: "a0bc3e29c32c864c41d1d49843018b6f"
server_key_version: "encryptionkey@0"
I20260812 04:01:42.749970 22859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "699da80645f149308aa0b17c0d4076a8"
format_stamp: "Formatted at 2026-08-12 04:01:42 on dist-test-slave-8tdl"
server_key: "7cb6481983d6cab5bfdc26c4492a5ba7"
server_key_iv: "a0bc3e29c32c864c41d1d49843018b6f"
server_key_version: "encryptionkey@0"
I20260812 04:01:42.751596 22859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:42.752473 22878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:42.752728 22859 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:42.752843 22859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "699da80645f149308aa0b17c0d4076a8"
format_stamp: "Formatted at 2026-08-12 04:01:42 on dist-test-slave-8tdl"
server_key: "7cb6481983d6cab5bfdc26c4492a5ba7"
server_key_iv: "a0bc3e29c32c864c41d1d49843018b6f"
server_key_version: "encryptionkey@0"
I20260812 04:01:42.752931 22859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:42.778074 22859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:42.780658 22859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:42.780848 22859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:42.785596 22859 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33709
I20260812 04:01:42.785676 22939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33709 every 8 connection(s)
I20260812 04:01:42.786062 22859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:42.787132 22940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:42.789580 22940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8: Bootstrap starting.
I20260812 04:01:42.790176 22940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:42.790452 22940 log.cc:824] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:42.791299 22940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8: No bootstrap required, opened a new log
I20260812 04:01:42.792970 22940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "699da80645f149308aa0b17c0d4076a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33709 } }
I20260812 04:01:42.793125 22940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:42.793149 22940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 699da80645f149308aa0b17c0d4076a8, State: Initialized, Role: FOLLOWER
I20260812 04:01:42.793303 22940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "699da80645f149308aa0b17c0d4076a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33709 } }
I20260812 04:01:42.793390 22940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:42.793416 22940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:42.793447 22940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:42.794023 22940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "699da80645f149308aa0b17c0d4076a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33709 } }
I20260812 04:01:42.794133 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 22859
I20260812 04:01:42.794154 22940 leader_election.cc:302] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 699da80645f149308aa0b17c0d4076a8; no voters:
I20260812 04:01:42.794258 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:42.794350 22940 leader_election.cc:288] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:42.794430 22944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:42.794489 31919 external_mini_cluster.cc:1428] Setting key 569c6233a9fce09f95f60cee6300718d
I20260812 04:01:42.794634 22944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [term 1 LEADER]: Becoming Leader. State: Replica: 699da80645f149308aa0b17c0d4076a8, State: Running, Role: LEADER
I20260812 04:01:42.794798 22944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "699da80645f149308aa0b17c0d4076a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33709 } }
I20260812 04:01:42.795269 22945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "699da80645f149308aa0b17c0d4076a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "699da80645f149308aa0b17c0d4076a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33709 } } }
I20260812 04:01:42.795383 22946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 699da80645f149308aa0b17c0d4076a8. Latest consensus state: current_term: 1 leader_uuid: "699da80645f149308aa0b17c0d4076a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "699da80645f149308aa0b17c0d4076a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33709 } } }
I20260812 04:01:42.795490 22946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:42.795426 22945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:42.794652 22940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:42.795719 22950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:42.796902 22950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:42.798556 22950 catalog_manager.cc:1347] Generated new cluster ID: 352ff78828d246ffba9c750d20a93ebd
I20260812 04:01:42.798609 22950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 04:01:42 dist-test-slave-8tdl krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 04:01:42.807667 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:42.794988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37417 (local address 127.0.0.1:33709)
0812 04:01:42.795338 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:42.795347 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:42.795383 (+ 36us) server_negotiation.cc:407] Connection header received
0812 04:01:42.795521 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:42.795529 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:42.795594 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:42.796405 (+ 811us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:42.796575 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:42.798306 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:42.799446 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:42.800065 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:42.802021 (+ 1956us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:42.803132 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:42.803136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:42.803137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:42.803149 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:42.805851 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:42.806264 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:42.806268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:42.806269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:42.806413 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:42.806627 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:42.806628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:42.806628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:42.806990 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:42.807201 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:42.807281 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 04:01:42.807431 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":91,"threads_started":1}
I20260812 04:01:42.808983 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:42.817268 22950 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:42.817924 22950 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:42.818373 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:42.810630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58959 (local address 127.0.0.1:33709)
0812 04:01:42.810668 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:42.810669 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:42.810966 (+ 297us) server_negotiation.cc:407] Connection header received
0812 04:01:42.810995 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:42.810996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:42.811013 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:42.811081 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:42.811671 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:42.812468 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:42.813460 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:42.813783 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:42.813855 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:42.815350 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:42.815352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:42.815352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:42.815360 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:42.817010 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:42.817443 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:42.817444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:42.817445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:42.817507 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:42.817736 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:42.817737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:42.817737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:42.817902 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:42.818020 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:42.818164 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 04:01:42.818278 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":18}
I20260812 04:01:42.822576 22950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 699da80645f149308aa0b17c0d4076a8: Generated new TSK 0
I20260812 04:01:42.822815 22950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:42.828313 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 22859
2026-08-12T04:01:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:42 dist-test-slave-8tdl krb5kdc[22973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:42 dist-test-slave-8tdl krb5kdc[22973](info): set up 2 sockets
Aug 12 04:01:42 dist-test-slave-8tdl krb5kdc[22973](info): commencing operation
krb5kdc: starting...
W20260812 04:01:43.967346 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:44 dist-test-slave-8tdl krb5kdc[22973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 04:01:44.106130 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39707
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:44.192675 22989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:44.192934 22989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:44.192977 22989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:44.194188 22989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:44.194263 22989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:44.194288 22989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:44.194315 22989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:44.194346 22989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:44.195902 22989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:44.196189 22989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:44.196439 22989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:44.202136 22998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:44.202140 22995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:44.202159 22996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:44.202309 22989 server_base.cc:1034] running on GCE node
I20260812 04:01:44.202745 22989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:44.203104 22989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:44.204276 22989 hybrid_clock.cc:648] HybridClock initialized: now 1786507304204267 us; error 48 us; skew 500 ppm
Aug 12 04:01:44 dist-test-slave-8tdl krb5kdc[22973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507304, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:44.206955 22989 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 04:01:44.207434 22989 webserver.cc:460] Webserver started at http://127.0.0.1:35131/ using document root <none> and password file <none>
I20260812 04:01:44.207669 22989 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:44.207737 22989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:44.207892 22989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:44.208969 22989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "c9ac570cb07843ff896bc90706200d3c"
format_stamp: "Formatted at 2026-08-12 04:01:44 on dist-test-slave-8tdl"
server_key: "4f13549b260791ac133017269a2e5b2a"
server_key_iv: "4de16d18c79b45a1ce53d81994a7477b"
server_key_version: "encryptionkey@0"
I20260812 04:01:44.209362 22989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "c9ac570cb07843ff896bc90706200d3c"
format_stamp: "Formatted at 2026-08-12 04:01:44 on dist-test-slave-8tdl"
server_key: "4f13549b260791ac133017269a2e5b2a"
server_key_iv: "4de16d18c79b45a1ce53d81994a7477b"
server_key_version: "encryptionkey@0"
I20260812 04:01:44.210804 22989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 04:01:44.211709 23005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:44.211928 22989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:44.212090 22989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "c9ac570cb07843ff896bc90706200d3c"
format_stamp: "Formatted at 2026-08-12 04:01:44 on dist-test-slave-8tdl"
server_key: "4f13549b260791ac133017269a2e5b2a"
server_key_iv: "4de16d18c79b45a1ce53d81994a7477b"
server_key_version: "encryptionkey@0"
I20260812 04:01:44.212180 22989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:44.235081 22989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:44.237316 22989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:44.237601 22989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:44.242111 22989 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39707
I20260812 04:01:44.242167 23065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39707 every 8 connection(s)
I20260812 04:01:44.242604 22989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:44.243556 23066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:44.245908 23066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c: Bootstrap starting.
I20260812 04:01:44.246443 23066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:44.246732 23066 log.cc:824] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:44.247574 23066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c: No bootstrap required, opened a new log
I20260812 04:01:44.249267 23066 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ac570cb07843ff896bc90706200d3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39707 } }
I20260812 04:01:44.249425 23066 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:44.249449 23066 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ac570cb07843ff896bc90706200d3c, State: Initialized, Role: FOLLOWER
I20260812 04:01:44.249614 23066 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ac570cb07843ff896bc90706200d3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39707 } }
I20260812 04:01:44.249676 23066 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:44.249718 23066 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:44.249769 23066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:44.250312 23066 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ac570cb07843ff896bc90706200d3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39707 } }
I20260812 04:01:44.250411 23066 leader_election.cc:302] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9ac570cb07843ff896bc90706200d3c; no voters:
I20260812 04:01:44.250603 23066 leader_election.cc:288] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:44.250650 23069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:44.250919 23069 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [term 1 LEADER]: Becoming Leader. State: Replica: c9ac570cb07843ff896bc90706200d3c, State: Running, Role: LEADER
I20260812 04:01:44.250927 23066 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:44.251111 23069 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ac570cb07843ff896bc90706200d3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39707 } }
I20260812 04:01:44.252161 23071 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9ac570cb07843ff896bc90706200d3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ac570cb07843ff896bc90706200d3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39707 } } }
I20260812 04:01:44.252271 23071 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:44.252161 23072 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9ac570cb07843ff896bc90706200d3c. Latest consensus state: current_term: 1 leader_uuid: "c9ac570cb07843ff896bc90706200d3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ac570cb07843ff896bc90706200d3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39707 } } }
I20260812 04:01:44.252473 23072 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:44.252630 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 22989
I20260812 04:01:44.252795 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
W20260812 04:01:44.252938 23086 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:01:44.252974 23086 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:01:44.253033 31919 external_mini_cluster.cc:1428] Setting key 65397eb10c2dbb86391a3d0cb0047100
I20260812 04:01:44.253124 23085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:44.253350 23085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:44.255038 23085 catalog_manager.cc:1347] Generated new cluster ID: e30f517a4fff465585072eb347958803
I20260812 04:01:44.255096 23085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:44.258533 23089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:44.253526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37269 (local address 127.0.0.1:39707)
0812 04:01:44.253708 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:44.253717 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:44.253921 (+ 204us) server_negotiation.cc:407] Connection header received
0812 04:01:44.254055 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:44.254063 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:44.254118 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:44.254268 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:44.254542 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:44.255701 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:44.256510 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:44.256931 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:44.257069 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:44.257799 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:44.257803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:44.257803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:44.257993 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:44.258032 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:44.258043 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:44.258123 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:44.258201 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 04:01:44.258362 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260812 04:01:44.259850 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:44.265717 23089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:44.261272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57021 (local address 127.0.0.1:39707)
0812 04:01:44.261309 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:44.261311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:44.262879 (+ 1568us) server_negotiation.cc:407] Connection header received
0812 04:01:44.262957 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:44.262958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:44.262969 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:44.263019 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:44.263161 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:44.263656 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:44.264441 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:44.264652 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:44.264730 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:44.265439 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:44.265440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:44.265440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:44.265446 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:44.265461 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:44.265465 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:44.265536 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:44.265599 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 04:01:44.265626 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 04:01:44.281344 23085 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:44.282006 23085 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:44.286785 23085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9ac570cb07843ff896bc90706200d3c: Generated new TSK 0
I20260812 04:01:44.286991 23085 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:44.294746 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 22989
2026-08-12T04:01:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1465 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:44 dist-test-slave-8tdl krb5kdc[23098](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:44 dist-test-slave-8tdl krb5kdc[23098](info): set up 2 sockets
Aug 12 04:01:44 dist-test-slave-8tdl krb5kdc[23098](info): commencing operation
krb5kdc: starting...
W20260812 04:01:45.423606 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:45 dist-test-slave-8tdl krb5kdc[23098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 04:01:45.566715 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41027
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:45.655385 23111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:45.655599 23111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:45.655622 23111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:45.656832 23111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:45.656881 23111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:45.656896 23111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:45.656908 23111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:45.656920 23111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:45.658496 23111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:45.658751 23111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:45.658983 23111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:45.664058 23117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:45.664103 23121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:45.664114 23119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:45.664343 23111 server_base.cc:1034] running on GCE node
I20260812 04:01:45.664628 23111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:45.664933 23111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:45.666137 23111 hybrid_clock.cc:648] HybridClock initialized: now 1786507305666115 us; error 52 us; skew 500 ppm
Aug 12 04:01:45 dist-test-slave-8tdl krb5kdc[23098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507305, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:45.669008 23111 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 04:01:45.669544 23111 webserver.cc:460] Webserver started at http://127.0.0.1:40393/ using document root <none> and password file <none>
I20260812 04:01:45.669792 23111 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:45.669875 23111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:45.670039 23111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:45.671245 23111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "c3028bab1d704bf5af3d8ebbe4d03943"
format_stamp: "Formatted at 2026-08-12 04:01:45 on dist-test-slave-8tdl"
server_key: "7a011ab027dc088f05d706bd87efd5e0"
server_key_iv: "55f5e0733f903c9511e565921cbbfa0c"
server_key_version: "encryptionkey@0"
I20260812 04:01:45.671631 23111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "c3028bab1d704bf5af3d8ebbe4d03943"
format_stamp: "Formatted at 2026-08-12 04:01:45 on dist-test-slave-8tdl"
server_key: "7a011ab027dc088f05d706bd87efd5e0"
server_key_iv: "55f5e0733f903c9511e565921cbbfa0c"
server_key_version: "encryptionkey@0"
I20260812 04:01:45.673086 23111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:01:45.673976 23130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:45.674190 23111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:45.674304 23111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "c3028bab1d704bf5af3d8ebbe4d03943"
format_stamp: "Formatted at 2026-08-12 04:01:45 on dist-test-slave-8tdl"
server_key: "7a011ab027dc088f05d706bd87efd5e0"
server_key_iv: "55f5e0733f903c9511e565921cbbfa0c"
server_key_version: "encryptionkey@0"
I20260812 04:01:45.674386 23111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:45.677973 23111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:45.680313 23111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:45.680514 23111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:45.685333 23111 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41027
I20260812 04:01:45.685405 23188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41027 every 8 connection(s)
I20260812 04:01:45.685779 23111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:45.686851 23189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:45.689203 23189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943: Bootstrap starting.
I20260812 04:01:45.689806 23189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:45.690127 23189 log.cc:824] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:45.691020 23189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943: No bootstrap required, opened a new log
I20260812 04:01:45.692659 23189 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3028bab1d704bf5af3d8ebbe4d03943" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41027 } }
I20260812 04:01:45.692807 23189 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:45.692830 23189 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3028bab1d704bf5af3d8ebbe4d03943, State: Initialized, Role: FOLLOWER
I20260812 04:01:45.693061 23189 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3028bab1d704bf5af3d8ebbe4d03943" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41027 } }
I20260812 04:01:45.693178 23189 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:45.693223 23189 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:45.693270 23189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:45.693257 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 23111
I20260812 04:01:45.693370 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:45.693591 31919 external_mini_cluster.cc:1428] Setting key 502b309a0df622a52ffd2c97adc5ffca
I20260812 04:01:45.693995 23189 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3028bab1d704bf5af3d8ebbe4d03943" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41027 } }
I20260812 04:01:45.694128 23189 leader_election.cc:302] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3028bab1d704bf5af3d8ebbe4d03943; no voters:
I20260812 04:01:45.694309 23189 leader_election.cc:288] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:45.694352 23194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:45.694541 23194 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [term 1 LEADER]: Becoming Leader. State: Replica: c3028bab1d704bf5af3d8ebbe4d03943, State: Running, Role: LEADER
I20260812 04:01:45.694644 23189 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:45.694770 23194 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3028bab1d704bf5af3d8ebbe4d03943" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41027 } }
I20260812 04:01:45.695236 23195 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3028bab1d704bf5af3d8ebbe4d03943" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3028bab1d704bf5af3d8ebbe4d03943" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41027 } } }
I20260812 04:01:45.695323 23195 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:45.695572 23196 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3028bab1d704bf5af3d8ebbe4d03943. Latest consensus state: current_term: 1 leader_uuid: "c3028bab1d704bf5af3d8ebbe4d03943" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3028bab1d704bf5af3d8ebbe4d03943" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41027 } } }
I20260812 04:01:45.695708 23196 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:45.695894 23198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:45.696193 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:45.694106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45751 (local address 127.0.0.1:41027)
0812 04:01:45.694342 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:45.694352 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:45.694391 (+ 39us) server_negotiation.cc:407] Connection header received
0812 04:01:45.694518 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:45.694526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:45.694558 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:45.695078 (+ 520us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:45.695577 (+ 499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:45.695580 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:45.695581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:45.695585 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:45.695630 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:45.695644 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:45.695932 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:45.695964 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 04:01:45.696005 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260812 04:01:45.696697 23198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:45.697351 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:45.698619 23198 catalog_manager.cc:1347] Generated new cluster ID: 36b3bb125c74465383093b5a4a19e448
I20260812 04:01:45.698666 23198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:45.703997 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:45.702881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43525 (local address 127.0.0.1:41027)
0812 04:01:45.702941 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:45.702943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:45.702953 (+ 10us) server_negotiation.cc:407] Connection header received
0812 04:01:45.702978 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:45.702979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:45.702983 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:45.703045 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:45.703790 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:45.703791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:45.703791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:45.703794 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:45.703807 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:45.703811 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:45.703852 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:45.703916 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 04:01:45.703929 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 04:01:45.706324 23198 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:45.706503 23198 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:45.712953 23198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3028bab1d704bf5af3d8ebbe4d03943: Generated new TSK 0
I20260812 04:01:45.713220 23198 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:45.715415 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 23111
2026-08-12T04:01:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:45 dist-test-slave-8tdl krb5kdc[23223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:45 dist-test-slave-8tdl krb5kdc[23223](info): set up 2 sockets
Aug 12 04:01:45 dist-test-slave-8tdl krb5kdc[23223](info): commencing operation
krb5kdc: starting...
W20260812 04:01:46.865459 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:46 dist-test-slave-8tdl krb5kdc[23223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 04:01:47.011546 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34113
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:47.103293 23236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:47.103529 23236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:47.103556 23236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:47.104805 23236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:47.104883 23236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:47.104907 23236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:47.104928 23236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:47.104984 23236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:47.106647 23236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:47.107019 23236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:47.107285 23236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:47.112748 23241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:47.112725 23244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:47.112761 23242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:47.112872 23236 server_base.cc:1034] running on GCE node
I20260812 04:01:47.113334 23236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:47.113658 23236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:47.114818 23236 hybrid_clock.cc:648] HybridClock initialized: now 1786507307114793 us; error 40 us; skew 500 ppm
Aug 12 04:01:47 dist-test-slave-8tdl krb5kdc[23223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507307, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:47.122603 23236 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 04:01:47.123198 23236 webserver.cc:460] Webserver started at http://127.0.0.1:45187/ using document root <none> and password file <none>
I20260812 04:01:47.123471 23236 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:47.123560 23236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:47.123732 23236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:47.124936 23236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "385e7477ce7b4506972a530f5597581e"
format_stamp: "Formatted at 2026-08-12 04:01:47 on dist-test-slave-8tdl"
server_key: "c826cea8f45c7784e4de3f720105c161"
server_key_iv: "4035edf4c6631751acffd85cd9fadfb7"
server_key_version: "encryptionkey@0"
I20260812 04:01:47.125350 23236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "385e7477ce7b4506972a530f5597581e"
format_stamp: "Formatted at 2026-08-12 04:01:47 on dist-test-slave-8tdl"
server_key: "c826cea8f45c7784e4de3f720105c161"
server_key_iv: "4035edf4c6631751acffd85cd9fadfb7"
server_key_version: "encryptionkey@0"
I20260812 04:01:47.126972 23236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:01:47.127892 23253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:47.128103 23236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:47.128235 23236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "385e7477ce7b4506972a530f5597581e"
format_stamp: "Formatted at 2026-08-12 04:01:47 on dist-test-slave-8tdl"
server_key: "c826cea8f45c7784e4de3f720105c161"
server_key_iv: "4035edf4c6631751acffd85cd9fadfb7"
server_key_version: "encryptionkey@0"
I20260812 04:01:47.128322 23236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:47.150476 23236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:47.153299 23236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:47.153508 23236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:47.158366 23236 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34113
I20260812 04:01:47.158473 23312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34113 every 8 connection(s)
I20260812 04:01:47.158960 23236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:47.160040 23313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:47.162559 23313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e: Bootstrap starting.
I20260812 04:01:47.163216 23313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:47.163533 23313 log.cc:824] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:47.164328 23313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e: No bootstrap required, opened a new log
I20260812 04:01:47.165993 23313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385e7477ce7b4506972a530f5597581e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } }
I20260812 04:01:47.166142 23313 raft_consensus.cc:374] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:47.166165 23313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 385e7477ce7b4506972a530f5597581e, State: Initialized, Role: FOLLOWER
I20260812 04:01:47.166359 23313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385e7477ce7b4506972a530f5597581e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } }
I20260812 04:01:47.166445 23313 raft_consensus.cc:388] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:47.166487 23313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:47.166548 23313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:47.167129 23313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385e7477ce7b4506972a530f5597581e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } }
I20260812 04:01:47.167222 23313 leader_election.cc:302] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 385e7477ce7b4506972a530f5597581e; no voters:
I20260812 04:01:47.167418 23313 leader_election.cc:288] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:47.167514 23317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:47.167695 23317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [term 1 LEADER]: Becoming Leader. State: Replica: 385e7477ce7b4506972a530f5597581e, State: Running, Role: LEADER
I20260812 04:01:47.167701 23313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:47.167801 23317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385e7477ce7b4506972a530f5597581e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } }
I20260812 04:01:47.168220 23319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 385e7477ce7b4506972a530f5597581e. Latest consensus state: current_term: 1 leader_uuid: "385e7477ce7b4506972a530f5597581e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385e7477ce7b4506972a530f5597581e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } } }
I20260812 04:01:47.168334 23319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:47.168217 23318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "385e7477ce7b4506972a530f5597581e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385e7477ce7b4506972a530f5597581e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34113 } } }
I20260812 04:01:47.168458 23318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:47.168641 23322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:47.169057 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 23236
I20260812 04:01:47.169209 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:47.169409 31919 external_mini_cluster.cc:1428] Setting key e20ce482de765daecef415582b2feb4b
I20260812 04:01:47.170150 23322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:47.172092 23322 catalog_manager.cc:1347] Generated new cluster ID: b1681a1b477448eca900492cc5c7da82
I20260812 04:01:47.172154 23322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 04:01:47 dist-test-slave-8tdl krb5kdc[23223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 04:01:47.182751 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:47.170284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:50075 (local address 127.0.0.1:34113)
0812 04:01:47.170762 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:47.170772 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:47.170818 (+ 46us) server_negotiation.cc:407] Connection header received
0812 04:01:47.170936 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:47.170944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:47.171006 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:47.171181 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:47.171958 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:47.173437 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:47.174559 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:47.175211 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:47.175309 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:47.178518 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:47.178523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:47.178523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:47.178537 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:47.180624 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:47.181147 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:47.181150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:47.181150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:47.181264 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:47.181529 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:47.181529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:47.181530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:47.181855 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:47.182021 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:47.182156 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 04:01:47.182397 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":79,"threads_started":1}
I20260812 04:01:47.184051 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:47.184118 23322 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:47.184561 23322 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:47.193459 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:47.185554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:41987 (local address 127.0.0.1:34113)
0812 04:01:47.185670 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:47.185673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:47.185737 (+ 64us) server_negotiation.cc:407] Connection header received
0812 04:01:47.185816 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:47.185817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:47.185828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:47.185878 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:47.186504 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:47.187322 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:47.188391 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:47.188712 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:47.188771 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:47.190402 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:47.190404 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:47.190404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:47.190410 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:47.192052 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:47.192504 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:47.192505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:47.192505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:47.192563 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:47.192799 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:47.192800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:47.192800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:47.192926 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:47.193018 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:47.193188 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 04:01:47.193252 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260812 04:01:47.194546 23322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 385e7477ce7b4506972a530f5597581e: Generated new TSK 0
I20260812 04:01:47.194798 23322 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:47.199435 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 23236
W20260812 04:01:47.208918 23221 connection.cc:535] client connection to 127.0.0.1:34113 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34113): Connection reset by peer (error 104)
2026-08-12T04:01:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:47 dist-test-slave-8tdl krb5kdc[23347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:47 dist-test-slave-8tdl krb5kdc[23347](info): set up 2 sockets
Aug 12 04:01:47 dist-test-slave-8tdl krb5kdc[23347](info): commencing operation
krb5kdc: starting...
W20260812 04:01:48.333755 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:48 dist-test-slave-8tdl krb5kdc[23347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 04:01:48.471066 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44349
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:48.560428 23360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:48.560662 23360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:48.560688 23360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:48.561898 23360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:48.561949 23360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:48.561965 23360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:48.561978 23360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:48.561990 23360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:48.563624 23360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:48.563903 23360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:48.564160 23360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:48.569672 23370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:48.569728 23367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:48.569689 23365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:48.569912 23360 server_base.cc:1034] running on GCE node
I20260812 04:01:48.570210 23360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:48.570541 23360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:48.571717 23360 hybrid_clock.cc:648] HybridClock initialized: now 1786507308571690 us; error 43 us; skew 500 ppm
Aug 12 04:01:48 dist-test-slave-8tdl krb5kdc[23347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507308, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:48.574427 23360 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 04:01:48.575006 23360 webserver.cc:460] Webserver started at http://127.0.0.1:35791/ using document root <none> and password file <none>
I20260812 04:01:48.575263 23360 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:48.575333 23360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:48.575480 23360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:48.576634 23360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "d53e741695b24aee99376994d5a4597b"
format_stamp: "Formatted at 2026-08-12 04:01:48 on dist-test-slave-8tdl"
server_key: "6dfd046db294de56ed7c7c17c671d697"
server_key_iv: "bd91af939eb18c2cd9b1c9237607b311"
server_key_version: "encryptionkey@0"
I20260812 04:01:48.577095 23360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "d53e741695b24aee99376994d5a4597b"
format_stamp: "Formatted at 2026-08-12 04:01:48 on dist-test-slave-8tdl"
server_key: "6dfd046db294de56ed7c7c17c671d697"
server_key_iv: "bd91af939eb18c2cd9b1c9237607b311"
server_key_version: "encryptionkey@0"
I20260812 04:01:48.578824 23360 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 04:01:48.579793 23379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:48.580077 23360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:48.580194 23360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "d53e741695b24aee99376994d5a4597b"
format_stamp: "Formatted at 2026-08-12 04:01:48 on dist-test-slave-8tdl"
server_key: "6dfd046db294de56ed7c7c17c671d697"
server_key_iv: "bd91af939eb18c2cd9b1c9237607b311"
server_key_version: "encryptionkey@0"
I20260812 04:01:48.580284 23360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:48.593503 23360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:48.595968 23360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:48.596161 23360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:48.601388 23360 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44349
I20260812 04:01:48.601430 23438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44349 every 8 connection(s)
I20260812 04:01:48.601850 23360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:48.602824 23439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:48.605217 23439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b: Bootstrap starting.
I20260812 04:01:48.605792 23439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:48.606074 23439 log.cc:824] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:48.606900 23439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b: No bootstrap required, opened a new log
I20260812 04:01:48.607592 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 23360
I20260812 04:01:48.607750 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:48.607967 31919 external_mini_cluster.cc:1428] Setting key 47d72e4798bef47cc756563dec5bfcbd
I20260812 04:01:48.609310 23439 raft_consensus.cc:348] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d53e741695b24aee99376994d5a4597b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } }
I20260812 04:01:48.609504 23439 raft_consensus.cc:374] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:48.609547 23439 raft_consensus.cc:729] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53e741695b24aee99376994d5a4597b, State: Initialized, Role: FOLLOWER
I20260812 04:01:48.609714 23439 consensus_queue.cc:260] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d53e741695b24aee99376994d5a4597b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } }
I20260812 04:01:48.609817 23439 raft_consensus.cc:388] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:48.609849 23439 raft_consensus.cc:482] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:48.609886 23439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:48.610741 23439 raft_consensus.cc:504] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d53e741695b24aee99376994d5a4597b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } }
I20260812 04:01:48.610857 23439 leader_election.cc:302] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d53e741695b24aee99376994d5a4597b; no voters:
I20260812 04:01:48.611109 23439 leader_election.cc:288] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:48.611228 23444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:48.611428 23439 sys_catalog.cc:564] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:48.611462 23444 raft_consensus.cc:686] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [term 1 LEADER]: Becoming Leader. State: Replica: d53e741695b24aee99376994d5a4597b, State: Running, Role: LEADER
I20260812 04:01:48.611635 23444 consensus_queue.cc:237] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d53e741695b24aee99376994d5a4597b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } }
I20260812 04:01:48.612242 23447 sys_catalog.cc:455] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d53e741695b24aee99376994d5a4597b. Latest consensus state: current_term: 1 leader_uuid: "d53e741695b24aee99376994d5a4597b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d53e741695b24aee99376994d5a4597b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } } }
I20260812 04:01:48.612414 23447 sys_catalog.cc:458] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:48.613060 23456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:48.613032 23445 sys_catalog.cc:455] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d53e741695b24aee99376994d5a4597b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d53e741695b24aee99376994d5a4597b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } } }
I20260812 04:01:48.613147 23445 sys_catalog.cc:458] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:48.613646 23456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:48.615448 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:48.608618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:42935 (local address 127.0.0.1:44349)
0812 04:01:48.608832 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:48.608843 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:48.608882 (+ 39us) server_negotiation.cc:407] Connection header received
0812 04:01:48.609259 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:48.609268 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:48.609353 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:48.609519 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:48.609831 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:48.611660 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:48.612887 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:48.613537 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:48.613657 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:48.614539 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:48.614543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:48.614544 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:48.614820 (+ 276us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:48.614857 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:48.614883 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:48.614997 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:48.615187 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 04:01:48.615220 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260812 04:01:48.615559 23456 catalog_manager.cc:1347] Generated new cluster ID: 555ac385a4964472a5e3cfdd7593dd1c
I20260812 04:01:48.615607 23456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:48.616555 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:48.623128 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:48.619193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:56563 (local address 127.0.0.1:44349)
0812 04:01:48.619232 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:48.619234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:48.619530 (+ 296us) server_negotiation.cc:407] Connection header received
0812 04:01:48.619581 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:48.619582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:48.619604 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:48.619667 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:48.619802 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:48.620571 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:48.621587 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:48.621927 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:48.622017 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:48.622813 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:48.622814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:48.622815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:48.622819 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:48.622834 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:48.622838 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:48.622867 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:48.623017 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 04:01:48.623033 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 04:01:48.641676 23456 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:48.642277 23456 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:48.646706 23456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d53e741695b24aee99376994d5a4597b: Generated new TSK 0
I20260812 04:01:48.646935 23456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:48.651533 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 23360
W20260812 04:01:48.661895 23345 connection.cc:535] client connection to 127.0.0.1:44349 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44349): Connection reset by peer (error 104)
2026-08-12T04:01:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1453 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:48 dist-test-slave-8tdl krb5kdc[23471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:48 dist-test-slave-8tdl krb5kdc[23471](info): set up 2 sockets
Aug 12 04:01:48 dist-test-slave-8tdl krb5kdc[23471](info): commencing operation
krb5kdc: starting...
W20260812 04:01:49.774578 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:49 dist-test-slave-8tdl krb5kdc[23471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 04:01:49.913853 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36153
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:50.000438 23484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:50.000705 23484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:50.000749 23484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:50.002097 23484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:50.002177 23484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:50.002200 23484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:50.002224 23484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:50.002368 23484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:50.004211 23484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:50.004549 23484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:50.004844 23484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:50.010308 23490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:50.010324 23492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:50.010309 23489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:50.010579 23484 server_base.cc:1034] running on GCE node
I20260812 04:01:50.010936 23484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:50.011267 23484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:50.012434 23484 hybrid_clock.cc:648] HybridClock initialized: now 1786507310012413 us; error 43 us; skew 500 ppm
Aug 12 04:01:50 dist-test-slave-8tdl krb5kdc[23471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507310, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:50.015631 23484 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 04:01:50.016135 23484 webserver.cc:460] Webserver started at http://127.0.0.1:37521/ using document root <none> and password file <none>
I20260812 04:01:50.016410 23484 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:50.016489 23484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:50.016649 23484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:50.017861 23484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "08b1466b3c0e45418201700658c483e5"
format_stamp: "Formatted at 2026-08-12 04:01:50 on dist-test-slave-8tdl"
server_key: "cc4b97ce7b845788a41fda9c6e8e50b1"
server_key_iv: "6f1581f0616c925d0dec93674d79711c"
server_key_version: "encryptionkey@0"
I20260812 04:01:50.018290 23484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "08b1466b3c0e45418201700658c483e5"
format_stamp: "Formatted at 2026-08-12 04:01:50 on dist-test-slave-8tdl"
server_key: "cc4b97ce7b845788a41fda9c6e8e50b1"
server_key_iv: "6f1581f0616c925d0dec93674d79711c"
server_key_version: "encryptionkey@0"
I20260812 04:01:50.020035 23484 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 04:01:50.020924 23500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:50.021175 23484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:50.021306 23484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "08b1466b3c0e45418201700658c483e5"
format_stamp: "Formatted at 2026-08-12 04:01:50 on dist-test-slave-8tdl"
server_key: "cc4b97ce7b845788a41fda9c6e8e50b1"
server_key_iv: "6f1581f0616c925d0dec93674d79711c"
server_key_version: "encryptionkey@0"
I20260812 04:01:50.021373 23484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:50.024959 23484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:50.027251 23484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:50.027415 23484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:50.031930 23484 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36153
I20260812 04:01:50.031963 23560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36153 every 8 connection(s)
I20260812 04:01:50.032344 23484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:50.033492 23561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:50.036142 23561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5: Bootstrap starting.
I20260812 04:01:50.036835 23561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:50.037227 23561 log.cc:824] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:50.038094 23561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5: No bootstrap required, opened a new log
I20260812 04:01:50.039888 23561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b1466b3c0e45418201700658c483e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36153 } }
I20260812 04:01:50.040048 23561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:50.040082 23561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08b1466b3c0e45418201700658c483e5, State: Initialized, Role: FOLLOWER
I20260812 04:01:50.040241 23561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b1466b3c0e45418201700658c483e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36153 } }
I20260812 04:01:50.040356 23561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:50.040403 23561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:50.040460 23561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:50.040448 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 23484
I20260812 04:01:50.040591 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:50.040805 31919 external_mini_cluster.cc:1428] Setting key e661bde451ae7da28e35f0b644a47a9b
I20260812 04:01:50.041240 23561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b1466b3c0e45418201700658c483e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36153 } }
I20260812 04:01:50.041424 23561 leader_election.cc:302] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08b1466b3c0e45418201700658c483e5; no voters:
I20260812 04:01:50.041622 23561 leader_election.cc:288] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:50.041646 23566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:50.041903 23561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:50.041860 23566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [term 1 LEADER]: Becoming Leader. State: Replica: 08b1466b3c0e45418201700658c483e5, State: Running, Role: LEADER
I20260812 04:01:50.042138 23566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b1466b3c0e45418201700658c483e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36153 } }
I20260812 04:01:50.042546 23567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08b1466b3c0e45418201700658c483e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b1466b3c0e45418201700658c483e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36153 } } }
I20260812 04:01:50.042627 23567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:50.042780 23568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08b1466b3c0e45418201700658c483e5. Latest consensus state: current_term: 1 leader_uuid: "08b1466b3c0e45418201700658c483e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b1466b3c0e45418201700658c483e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36153 } } }
I20260812 04:01:50.042845 23568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:50.043339 23578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:50.043771 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:50.041356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:39381 (local address 127.0.0.1:36153)
0812 04:01:50.041634 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:50.041645 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:50.041681 (+ 36us) server_negotiation.cc:407] Connection header received
0812 04:01:50.041810 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:50.041818 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:50.041849 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:50.042845 (+ 996us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:50.043181 (+ 336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:50.043184 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:50.043185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:50.043188 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:50.043225 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:50.043240 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:50.043342 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:50.043530 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 04:01:50.043579 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":140,"threads_started":1}
I20260812 04:01:50.043932 23578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:50.044812 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:50.046342 23578 catalog_manager.cc:1347] Generated new cluster ID: 988499e6f27647a9be9949f96c08aec9
I20260812 04:01:50.046414 23578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:50.057988 23578 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:50.058234 23578 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:50.058794 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:50.046566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:60879 (local address 127.0.0.1:36153)
0812 04:01:50.046625 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:50.046627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:50.057595 (+ 10968us) server_negotiation.cc:407] Connection header received
0812 04:01:50.057711 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:50.057712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:50.057733 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:50.057793 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:50.058540 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:50.058541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:50.058542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:50.058548 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:50.058565 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:50.058570 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:50.058623 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:50.058703 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 04:01:50.058721 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 04:01:50.070397 23578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08b1466b3c0e45418201700658c483e5: Generated new TSK 0
I20260812 04:01:50.070712 23578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:50.078349 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 23484
2026-08-12T04:01:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:50 dist-test-slave-8tdl krb5kdc[23598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:50 dist-test-slave-8tdl krb5kdc[23598](info): set up 2 sockets
Aug 12 04:01:50 dist-test-slave-8tdl krb5kdc[23598](info): commencing operation
krb5kdc: starting...
W20260812 04:01:51.263441 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:51 dist-test-slave-8tdl krb5kdc[23598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 04:01:51.402047 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:51.488135 23611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:51.488382 23611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:51.488422 23611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:51.489562 23611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:51.489631 23611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:51.489656 23611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:51.489686 23611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:51.489718 23611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:51.491256 23611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:51.491539 23611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:51.491768 23611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:51.496856 23616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:51.496917 23611 server_base.cc:1034] running on GCE node
W20260812 04:01:51.496862 23617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:51.496862 23619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:51.497370 23611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:51.497716 23611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:51.498896 23611 hybrid_clock.cc:648] HybridClock initialized: now 1786507311498873 us; error 35 us; skew 500 ppm
Aug 12 04:01:51 dist-test-slave-8tdl krb5kdc[23598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507311, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:51.501542 23611 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 04:01:51.502076 23611 webserver.cc:460] Webserver started at http://127.0.0.1:42333/ using document root <none> and password file <none>
I20260812 04:01:51.502311 23611 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:51.502379 23611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:51.502640 23611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:51.504122 23611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "2ef64fa2168c4d79a45cf14120685df5"
format_stamp: "Formatted at 2026-08-12 04:01:51 on dist-test-slave-8tdl"
server_key: "f2b4a81d4325eff31b56f8975ff75eb6"
server_key_iv: "e45d399f048a3eb3963900593cd5bc06"
server_key_version: "encryptionkey@0"
I20260812 04:01:51.504611 23611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "2ef64fa2168c4d79a45cf14120685df5"
format_stamp: "Formatted at 2026-08-12 04:01:51 on dist-test-slave-8tdl"
server_key: "f2b4a81d4325eff31b56f8975ff75eb6"
server_key_iv: "e45d399f048a3eb3963900593cd5bc06"
server_key_version: "encryptionkey@0"
I20260812 04:01:51.510421 23611 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.001s
I20260812 04:01:51.511534 23626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:51.511737 23611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:51.511847 23611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "2ef64fa2168c4d79a45cf14120685df5"
format_stamp: "Formatted at 2026-08-12 04:01:51 on dist-test-slave-8tdl"
server_key: "f2b4a81d4325eff31b56f8975ff75eb6"
server_key_iv: "e45d399f048a3eb3963900593cd5bc06"
server_key_version: "encryptionkey@0"
I20260812 04:01:51.511935 23611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:51.526520 23611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:51.529065 23611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:51.529233 23611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:51.533812 23611 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39225
I20260812 04:01:51.533846 23681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39225 every 8 connection(s)
I20260812 04:01:51.534240 23611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:51.535265 23683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:51.537535 23683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5: Bootstrap starting.
I20260812 04:01:51.538100 23683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:51.538408 23683 log.cc:824] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:51.538492 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 23611
I20260812 04:01:51.538610 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:51.538872 31919 external_mini_cluster.cc:1428] Setting key d89e8237690fc5d9317cd2bd75dd749c
I20260812 04:01:51.539280 23683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5: No bootstrap required, opened a new log
I20260812 04:01:51.541508 23683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef64fa2168c4d79a45cf14120685df5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39225 } }
I20260812 04:01:51.541704 23683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:51.541740 23683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef64fa2168c4d79a45cf14120685df5, State: Initialized, Role: FOLLOWER
I20260812 04:01:51.541901 23683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef64fa2168c4d79a45cf14120685df5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39225 } }
I20260812 04:01:51.541997 23683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:51.542030 23683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:51.542075 23683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:51.542831 23683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef64fa2168c4d79a45cf14120685df5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39225 } }
I20260812 04:01:51.542943 23683 leader_election.cc:302] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ef64fa2168c4d79a45cf14120685df5; no voters:
I20260812 04:01:51.543114 23683 leader_election.cc:288] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:51.543177 23688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:51.543342 23688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [term 1 LEADER]: Becoming Leader. State: Replica: 2ef64fa2168c4d79a45cf14120685df5, State: Running, Role: LEADER
I20260812 04:01:51.543349 23683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:51.543525 23688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef64fa2168c4d79a45cf14120685df5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39225 } }
I20260812 04:01:51.543946 23691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ef64fa2168c4d79a45cf14120685df5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef64fa2168c4d79a45cf14120685df5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39225 } } }
I20260812 04:01:51.544026 23691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:51.544267 23689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ef64fa2168c4d79a45cf14120685df5. Latest consensus state: current_term: 1 leader_uuid: "2ef64fa2168c4d79a45cf14120685df5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef64fa2168c4d79a45cf14120685df5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39225 } } }
I20260812 04:01:51.544495 23689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:51.544672 23696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:51.545198 23696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 04:01:51 dist-test-slave-8tdl krb5kdc[23598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 04:01:51.546866 23696 catalog_manager.cc:1347] Generated new cluster ID: 7177e5390f724e95a485f1967cd52f13
I20260812 04:01:51.548233 23696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:51.552009 23687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:51.539399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:35549 (local address 127.0.0.1:39225)
0812 04:01:51.539613 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:51.539622 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:51.539657 (+ 35us) server_negotiation.cc:407] Connection header received
0812 04:01:51.540644 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:51.540654 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:51.540722 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:51.540882 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:51.541524 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:51.543263 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:51.544396 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:51.545023 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:51.545131 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:51.548164 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:51.548168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:51.548169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:51.548183 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:51.550210 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:51.550701 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:51.550704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:51.550704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:51.550811 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:51.551090 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:51.551091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:51.551091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:51.551379 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:51.551530 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:51.551690 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 04:01:51.551789 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
I20260812 04:01:51.552932 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:51.561831 23687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:51.554429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:53015 (local address 127.0.0.1:39225)
0812 04:01:51.554464 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:51.554466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:51.554826 (+ 360us) server_negotiation.cc:407] Connection header received
0812 04:01:51.554926 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:51.554926 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:51.554936 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:51.554968 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:51.555595 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:51.556346 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:51.557309 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:51.557633 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:51.557670 (+ 37us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:51.559127 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:51.559128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:51.559128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:51.559135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:51.560647 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:51.561044 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:51.561045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:51.561045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:51.561093 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:51.561288 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:51.561289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:51.561289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:51.561399 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:51.561470 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:51.561677 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 04:01:51.561738 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 04:01:51.563624 23696 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:51.564144 23696 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:51.579028 23696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ef64fa2168c4d79a45cf14120685df5: Generated new TSK 0
I20260812 04:01:51.579274 23696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:51.584180 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 23611
W20260812 04:01:51.595772 23596 connection.cc:535] client connection to 127.0.0.1:39225 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39225): Connection reset by peer (error 104)
2026-08-12T04:01:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:51 dist-test-slave-8tdl krb5kdc[23717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:51 dist-test-slave-8tdl krb5kdc[23717](info): set up 2 sockets
Aug 12 04:01:51 dist-test-slave-8tdl krb5kdc[23717](info): commencing operation
krb5kdc: starting...
W20260812 04:01:52.704649 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:52 dist-test-slave-8tdl krb5kdc[23717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 04:01:52.843374 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40451
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:52.933166 23733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:52.933434 23733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:52.933473 23733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:52.934706 23733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:52.934782 23733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:52.934804 23733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:52.934824 23733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:52.934856 23733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:52.936458 23733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40451
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:52.936754 23733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:52.937006 23733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:52.942165 23740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:52.942152 23739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:52.942194 23743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:52.942668 23733 server_base.cc:1034] running on GCE node
I20260812 04:01:52.942981 23733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:52.943320 23733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:52.944530 23733 hybrid_clock.cc:648] HybridClock initialized: now 1786507312944520 us; error 39 us; skew 500 ppm
Aug 12 04:01:52 dist-test-slave-8tdl krb5kdc[23717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507312, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:52.952598 23733 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 04:01:52.953143 23733 webserver.cc:460] Webserver started at http://127.0.0.1:39857/ using document root <none> and password file <none>
I20260812 04:01:52.953419 23733 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:52.953493 23733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:52.953655 23733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:52.954859 23733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "c9b2e4c7505f44c298bb89b994aa99f8"
format_stamp: "Formatted at 2026-08-12 04:01:52 on dist-test-slave-8tdl"
server_key: "220aaed0654121a5a295452334d30517"
server_key_iv: "900e24ee195da36c27efca9061eda585"
server_key_version: "encryptionkey@0"
I20260812 04:01:52.955257 23733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "c9b2e4c7505f44c298bb89b994aa99f8"
format_stamp: "Formatted at 2026-08-12 04:01:52 on dist-test-slave-8tdl"
server_key: "220aaed0654121a5a295452334d30517"
server_key_iv: "900e24ee195da36c27efca9061eda585"
server_key_version: "encryptionkey@0"
I20260812 04:01:52.956746 23733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 04:01:52.957561 23750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:52.957827 23733 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:52.957964 23733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "c9b2e4c7505f44c298bb89b994aa99f8"
format_stamp: "Formatted at 2026-08-12 04:01:52 on dist-test-slave-8tdl"
server_key: "220aaed0654121a5a295452334d30517"
server_key_iv: "900e24ee195da36c27efca9061eda585"
server_key_version: "encryptionkey@0"
I20260812 04:01:52.958061 23733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:52.980674 23733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:52.983510 23733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:52.983711 23733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:52.988794 23733 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40451
I20260812 04:01:52.988839 23804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40451 every 8 connection(s)
I20260812 04:01:52.989261 23733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:52.990222 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 23733
I20260812 04:01:52.990296 23805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:52.990392 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:52.990638 31919 external_mini_cluster.cc:1428] Setting key 082084fa4f6b0b8f88bf6f091ef92f3d
I20260812 04:01:52.993824 23805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8: Bootstrap starting.
I20260812 04:01:52.994590 23805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:52.994951 23805 log.cc:824] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:52.995674 23805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8: No bootstrap required, opened a new log
I20260812 04:01:52.996968 23809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:52.991316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:60195 (local address 127.0.0.1:40451)
0812 04:01:52.991512 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:52.991529 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:52.991557 (+ 28us) server_negotiation.cc:407] Connection header received
0812 04:01:52.992061 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:52.992068 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:52.992122 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:52.992246 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:52.992527 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:52.993468 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:52.994592 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:52.995177 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:52.995326 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:52.996111 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:52.996116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:52.996116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:52.996331 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:52.996363 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:52.996374 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:52.996471 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:52.996606 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 04:01:52.996638 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260812 04:01:52.997298 23805 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b2e4c7505f44c298bb89b994aa99f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40451 } }
I20260812 04:01:52.997478 23805 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:52.997520 23805 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9b2e4c7505f44c298bb89b994aa99f8, State: Initialized, Role: FOLLOWER
I20260812 04:01:52.997640 23805 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b2e4c7505f44c298bb89b994aa99f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40451 } }
I20260812 04:01:52.997740 23805 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:52.997784 23805 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:52.997828 23805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:52.998121 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:52.998502 23805 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b2e4c7505f44c298bb89b994aa99f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40451 } }
I20260812 04:01:52.998620 23805 leader_election.cc:302] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9b2e4c7505f44c298bb89b994aa99f8; no voters:
I20260812 04:01:52.998808 23805 leader_election.cc:288] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:52.998843 23811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:52.998963 23811 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [term 1 LEADER]: Becoming Leader. State: Replica: c9b2e4c7505f44c298bb89b994aa99f8, State: Running, Role: LEADER
I20260812 04:01:52.999078 23805 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:52.999099 23811 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b2e4c7505f44c298bb89b994aa99f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40451 } }
I20260812 04:01:52.999414 23811 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9b2e4c7505f44c298bb89b994aa99f8. Latest consensus state: current_term: 1 leader_uuid: "c9b2e4c7505f44c298bb89b994aa99f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b2e4c7505f44c298bb89b994aa99f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40451 } } }
I20260812 04:01:52.999428 23812 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9b2e4c7505f44c298bb89b994aa99f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b2e4c7505f44c298bb89b994aa99f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40451 } } }
I20260812 04:01:52.999601 23812 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:52.999580 23811 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:52.999965 23826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:53.000531 23826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:53.002476 23826 catalog_manager.cc:1347] Generated new cluster ID: cf00bc88b58f4a248fb06b75a99b3182
I20260812 04:01:53.002532 23826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:53.003325 23809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:52.999635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:35115 (local address 127.0.0.1:40451)
0812 04:01:52.999700 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:52.999702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:52.999954 (+ 252us) server_negotiation.cc:407] Connection header received
0812 04:01:53.000046 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:53.000047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:53.000059 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:53.000107 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:53.000268 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:53.001002 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:53.001885 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:53.002187 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:53.002257 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:53.003057 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:53.003058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:53.003059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:53.003063 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:53.003088 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:53.003093 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:53.003171 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:53.003237 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 04:01:53.003246 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 04:01:53.014307 23826 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:53.014986 23826 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:53.017377 23826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9b2e4c7505f44c298bb89b994aa99f8: Generated new TSK 0
I20260812 04:01:53.017599 23826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:53.027278 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 23733
W20260812 04:01:53.037333 23714 connection.cc:535] client connection to 127.0.0.1:40451 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40451): Connection reset by peer (error 104)
2026-08-12T04:01:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:53 dist-test-slave-8tdl krb5kdc[23841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:53 dist-test-slave-8tdl krb5kdc[23841](info): set up 2 sockets
Aug 12 04:01:53 dist-test-slave-8tdl krb5kdc[23841](info): commencing operation
krb5kdc: starting...
W20260812 04:01:54.176075 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:54 dist-test-slave-8tdl krb5kdc[23841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 04:01:54.314499 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36339
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:54.402949 23854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:54.403173 23854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:54.403198 23854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:54.404594 23854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:54.404662 23854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:54.404680 23854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:54.404692 23854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:54.404706 23854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:54.406561 23854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36339
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:54.406884 23854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:54.407186 23854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:54.412395 23859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:54.412520 23860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:54.412573 23863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:54.412741 23854 server_base.cc:1034] running on GCE node
I20260812 04:01:54.413131 23854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:54.413429 23854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:54.414579 23854 hybrid_clock.cc:648] HybridClock initialized: now 1786507314414563 us; error 28 us; skew 500 ppm
Aug 12 04:01:54 dist-test-slave-8tdl krb5kdc[23841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507314, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:54.417600 23854 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 04:01:54.418089 23854 webserver.cc:460] Webserver started at http://127.0.0.1:37601/ using document root <none> and password file <none>
I20260812 04:01:54.418323 23854 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:54.418372 23854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:54.418507 23854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:54.419711 23854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "b04d479e7e924de78fc2a6ab7a01b731"
format_stamp: "Formatted at 2026-08-12 04:01:54 on dist-test-slave-8tdl"
server_key: "24995ce503e92f8113ed59d353bae244"
server_key_iv: "91e4dcb918768c5b0be6a5fa1447354a"
server_key_version: "encryptionkey@0"
I20260812 04:01:54.420089 23854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "b04d479e7e924de78fc2a6ab7a01b731"
format_stamp: "Formatted at 2026-08-12 04:01:54 on dist-test-slave-8tdl"
server_key: "24995ce503e92f8113ed59d353bae244"
server_key_iv: "91e4dcb918768c5b0be6a5fa1447354a"
server_key_version: "encryptionkey@0"
I20260812 04:01:54.421561 23854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 04:01:54.422400 23871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:54.422622 23854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 04:01:54.422755 23854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "b04d479e7e924de78fc2a6ab7a01b731"
format_stamp: "Formatted at 2026-08-12 04:01:54 on dist-test-slave-8tdl"
server_key: "24995ce503e92f8113ed59d353bae244"
server_key_iv: "91e4dcb918768c5b0be6a5fa1447354a"
server_key_version: "encryptionkey@0"
I20260812 04:01:54.422827 23854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:54.426146 23854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:54.428196 23854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:54.428310 23854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:54.432657 23854 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36339
I20260812 04:01:54.432710 23926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36339 every 8 connection(s)
I20260812 04:01:54.433063 23854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:54.434192 23927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:54.436753 23927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731: Bootstrap starting.
I20260812 04:01:54.437386 23927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:54.437695 23927 log.cc:824] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:54.438417 23927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731: No bootstrap required, opened a new log
I20260812 04:01:54.440124 23927 raft_consensus.cc:348] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04d479e7e924de78fc2a6ab7a01b731" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36339 } }
I20260812 04:01:54.440289 23927 raft_consensus.cc:374] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:54.440336 23927 raft_consensus.cc:729] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b04d479e7e924de78fc2a6ab7a01b731, State: Initialized, Role: FOLLOWER
I20260812 04:01:54.440505 23927 consensus_queue.cc:260] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04d479e7e924de78fc2a6ab7a01b731" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36339 } }
I20260812 04:01:54.440587 23927 raft_consensus.cc:388] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:54.440631 23927 raft_consensus.cc:482] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:54.440680 23927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:54.441295 23927 raft_consensus.cc:504] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04d479e7e924de78fc2a6ab7a01b731" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36339 } }
I20260812 04:01:54.441346 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 23854
I20260812 04:01:54.441480 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:54.441432 23927 leader_election.cc:302] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b04d479e7e924de78fc2a6ab7a01b731; no voters:
I20260812 04:01:54.441705 23927 leader_election.cc:288] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:54.441733 31919 external_mini_cluster.cc:1428] Setting key 0eb376cf29c305ab39c773f97990c86e
I20260812 04:01:54.441766 23931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:54.442003 23927 sys_catalog.cc:564] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:54.442008 23931 raft_consensus.cc:686] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [term 1 LEADER]: Becoming Leader. State: Replica: b04d479e7e924de78fc2a6ab7a01b731, State: Running, Role: LEADER
I20260812 04:01:54.442256 23931 consensus_queue.cc:237] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04d479e7e924de78fc2a6ab7a01b731" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36339 } }
I20260812 04:01:54.442719 23933 sys_catalog.cc:455] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b04d479e7e924de78fc2a6ab7a01b731. Latest consensus state: current_term: 1 leader_uuid: "b04d479e7e924de78fc2a6ab7a01b731" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04d479e7e924de78fc2a6ab7a01b731" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36339 } } }
I20260812 04:01:54.442834 23932 sys_catalog.cc:455] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b04d479e7e924de78fc2a6ab7a01b731" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04d479e7e924de78fc2a6ab7a01b731" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36339 } } }
I20260812 04:01:54.442950 23932 sys_catalog.cc:458] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:54.443163 23933 sys_catalog.cc:458] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:54.443579 23938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:54.444208 23938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:54.444463 23935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:54.442362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:45543 (local address 127.0.0.1:36339)
0812 04:01:54.442562 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:54.442572 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:54.442609 (+ 37us) server_negotiation.cc:407] Connection header received
0812 04:01:54.442767 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:54.442774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:54.442806 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:54.443265 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:54.443793 (+ 528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:54.443797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:54.443797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:54.443803 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:54.443857 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:54.443875 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:54.443950 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:54.444047 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 04:01:54.444204 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260812 04:01:54.445703 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:54.445957 23938 catalog_manager.cc:1347] Generated new cluster ID: effbe9aa6ef04f29af346ac529a64fd1
I20260812 04:01:54.446007 23938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:54.448391 23935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:54.447253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:39185 (local address 127.0.0.1:36339)
0812 04:01:54.447285 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:54.447287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:54.447446 (+ 159us) server_negotiation.cc:407] Connection header received
0812 04:01:54.447508 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:54.447508 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:54.447513 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:54.447566 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:54.448192 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:54.448193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:54.448193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:54.448197 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:54.448209 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:54.448213 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:54.448255 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:54.448320 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 04:01:54.448331 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 04:01:54.452961 23938 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:54.453126 23938 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:54.465047 23938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b04d479e7e924de78fc2a6ab7a01b731: Generated new TSK 0
I20260812 04:01:54.465291 23938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:54.472663 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 23854
2026-08-12T04:01:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:54 dist-test-slave-8tdl krb5kdc[23963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:54 dist-test-slave-8tdl krb5kdc[23963](info): set up 2 sockets
Aug 12 04:01:54 dist-test-slave-8tdl krb5kdc[23963](info): commencing operation
krb5kdc: starting...
W20260812 04:01:55.627562 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:55 dist-test-slave-8tdl krb5kdc[23963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:01:55.770517 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:37419
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:37419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:55.858287 23977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:55.858511 23977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:55.858538 23977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:55.859938 23977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:55.860016 23977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:55.860033 23977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:55.860047 23977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:55.860060 23977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:55.861878 23977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:37419
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:55.862154 23977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:55.862344 23977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:55.867486 23982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:55.867486 23985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:55.867580 23983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:55.868134 23977 server_base.cc:1034] running on GCE node
I20260812 04:01:55.868435 23977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:55.868810 23977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:55.870033 23977 hybrid_clock.cc:648] HybridClock initialized: now 1786507315870002 us; error 49 us; skew 500 ppm
Aug 12 04:01:55 dist-test-slave-8tdl krb5kdc[23963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507315, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:55.873180 23977 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:01:55.873719 23977 webserver.cc:460] Webserver started at http://127.31.43.254:37271/ using document root <none> and password file <none>
I20260812 04:01:55.873945 23977 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:55.873992 23977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:55.874106 23977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:55.875334 23977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "884d43eb5e11447b9700242d918d5c98"
format_stamp: "Formatted at 2026-08-12 04:01:55 on dist-test-slave-8tdl"
server_key: "29a327dd20a3312c844734ed9de9edef"
server_key_iv: "122fbd23b9f8df6b97fb9f82a60ab803"
server_key_version: "encryptionkey@0"
I20260812 04:01:55.875686 23977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "884d43eb5e11447b9700242d918d5c98"
format_stamp: "Formatted at 2026-08-12 04:01:55 on dist-test-slave-8tdl"
server_key: "29a327dd20a3312c844734ed9de9edef"
server_key_iv: "122fbd23b9f8df6b97fb9f82a60ab803"
server_key_version: "encryptionkey@0"
I20260812 04:01:55.877206 23977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:01:55.878001 23992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:55.878233 23977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:55.878340 23977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "884d43eb5e11447b9700242d918d5c98"
format_stamp: "Formatted at 2026-08-12 04:01:55 on dist-test-slave-8tdl"
server_key: "29a327dd20a3312c844734ed9de9edef"
server_key_iv: "122fbd23b9f8df6b97fb9f82a60ab803"
server_key_version: "encryptionkey@0"
I20260812 04:01:55.878404 23977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:55.890358 23977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:55.893141 23977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:55.893278 23977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:55.898038 23977 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:37419
I20260812 04:01:55.898082 24055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:37419 every 8 connection(s)
I20260812 04:01:55.898471 23977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:55.899518 24056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:55.901975 24056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98: Bootstrap starting.
I20260812 04:01:55.902779 24056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:55.903096 24056 log.cc:824] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:55.903863 24056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98: No bootstrap required, opened a new log
I20260812 04:01:55.905630 24056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884d43eb5e11447b9700242d918d5c98" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37419 } }
I20260812 04:01:55.905781 24056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:55.905803 24056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884d43eb5e11447b9700242d918d5c98, State: Initialized, Role: FOLLOWER
I20260812 04:01:55.905973 24056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884d43eb5e11447b9700242d918d5c98" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37419 } }
I20260812 04:01:55.906039 24056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:55.906095 24056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:55.906149 24056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:55.907189 24056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884d43eb5e11447b9700242d918d5c98" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37419 } }
I20260812 04:01:55.907279 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 23977
I20260812 04:01:55.907323 24056 leader_election.cc:302] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 884d43eb5e11447b9700242d918d5c98; no voters:
I20260812 04:01:55.907397 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:55.907516 24056 leader_election.cc:288] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:55.907584 24059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:55.907653 31919 external_mini_cluster.cc:1428] Setting key 03890df70a891b06ae6d1ec7b7c3c7c5
I20260812 04:01:55.907763 24059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [term 1 LEADER]: Becoming Leader. State: Replica: 884d43eb5e11447b9700242d918d5c98, State: Running, Role: LEADER
I20260812 04:01:55.907819 24056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:55.907867 24059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884d43eb5e11447b9700242d918d5c98" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37419 } }
I20260812 04:01:55.908341 24060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "884d43eb5e11447b9700242d918d5c98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884d43eb5e11447b9700242d918d5c98" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37419 } } }
I20260812 04:01:55.908425 24060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:55.908659 24062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 884d43eb5e11447b9700242d918d5c98. Latest consensus state: current_term: 1 leader_uuid: "884d43eb5e11447b9700242d918d5c98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884d43eb5e11447b9700242d918d5c98" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 37419 } } }
I20260812 04:01:55.908768 24062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98 [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:55.908929 24070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:55.909806 24070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:55.911583 24070 catalog_manager.cc:1347] Generated new cluster ID: 9fd6b8c31f5847c2b59b169c1a231570
I20260812 04:01:55.911644 24070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 04:01:55 dist-test-slave-8tdl krb5kdc[23963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:01:55.920324 24068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:55.908377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34347 (local address 127.31.43.254:37419)
0812 04:01:55.908577 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:55.908587 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:55.908624 (+ 37us) server_negotiation.cc:407] Connection header received
0812 04:01:55.908751 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:55.908759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:55.908823 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:55.908990 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:55.910128 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:55.911409 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:55.912530 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:55.913212 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:55.913329 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:55.916308 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:55.916312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:55.916313 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:55.916327 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:55.918371 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:55.918911 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:55.918915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:55.918915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:55.919061 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:55.919320 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:55.919321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:55.919321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:55.919655 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:55.919810 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:55.919963 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 04:01:55.920080 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260812 04:01:55.921487 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:55.927608 24070 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:55.928311 24070 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:55.932539 24070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 884d43eb5e11447b9700242d918d5c98: Generated new TSK 0
I20260812 04:01:55.933326 24070 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:55.934284 24068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:55.923061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47989 (local address 127.31.43.254:37419)
0812 04:01:55.923102 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:55.923104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:55.926893 (+ 3789us) server_negotiation.cc:407] Connection header received
0812 04:01:55.926996 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:55.926997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:55.927009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:55.927039 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:55.927802 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:55.928495 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:55.929544 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:55.929857 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:55.929917 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:55.931512 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:55.931514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:55.931514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:01:55.931521 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:55.933062 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:55.933511 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:55.933512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:55.933513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:55.933579 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:01:55.933824 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:01:55.933825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:01:55.933825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:01:55.933927 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:55.934007 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:55.934171 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 04:01:55.934224 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 04:01:55.936851 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 23977
W20260812 04:01:55.944605 23961 connection.cc:535] client connection to 127.31.43.254:37419 recv error: Network error: failed to read from TLS socket (remote: 127.31.43.254:37419): Connection reset by peer (error 104)
2026-08-12T04:01:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:55 dist-test-slave-8tdl krb5kdc[24089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:55 dist-test-slave-8tdl krb5kdc[24089](info): set up 2 sockets
Aug 12 04:01:55 dist-test-slave-8tdl krb5kdc[24089](info): commencing operation
krb5kdc: starting...
W20260812 04:01:57.052379 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:57 dist-test-slave-8tdl krb5kdc[24089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:01:57.190426 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:36433
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:36433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:57.276685 24104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:57.276928 24104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:57.276955 24104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:57.278187 24104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:57.278240 24104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:57.278254 24104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:57.278267 24104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:57.278280 24104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:57.279958 24104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:36433
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:36433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:57.280257 24104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:57.280519 24104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:57.285820 24109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:57.285828 24112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:57.285889 24110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:57.286238 24104 server_base.cc:1034] running on GCE node
I20260812 04:01:57.286513 24104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:57.286846 24104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:57.288096 24104 hybrid_clock.cc:648] HybridClock initialized: now 1786507317288075 us; error 38 us; skew 500 ppm
Aug 12 04:01:57 dist-test-slave-8tdl krb5kdc[24089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507317, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:57.291075 24104 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:01:57.291584 24104 webserver.cc:460] Webserver started at http://127.31.43.254:34493/ using document root <none> and password file <none>
I20260812 04:01:57.291837 24104 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:57.291906 24104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:57.292057 24104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:57.293215 24104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "6b996b1e9baa451f9ff7ced26b3d1c1d"
format_stamp: "Formatted at 2026-08-12 04:01:57 on dist-test-slave-8tdl"
server_key: "1fb9ead9873bcb1a32ca31da8fa970c6"
server_key_iv: "1194f9af3be8568d880a69f1046c70ec"
server_key_version: "encryptionkey@0"
I20260812 04:01:57.293591 24104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "6b996b1e9baa451f9ff7ced26b3d1c1d"
format_stamp: "Formatted at 2026-08-12 04:01:57 on dist-test-slave-8tdl"
server_key: "1fb9ead9873bcb1a32ca31da8fa970c6"
server_key_iv: "1194f9af3be8568d880a69f1046c70ec"
server_key_version: "encryptionkey@0"
I20260812 04:01:57.295138 24104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 04:01:57.295925 24122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:57.296213 24104 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:01:57.296316 24104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "6b996b1e9baa451f9ff7ced26b3d1c1d"
format_stamp: "Formatted at 2026-08-12 04:01:57 on dist-test-slave-8tdl"
server_key: "1fb9ead9873bcb1a32ca31da8fa970c6"
server_key_iv: "1194f9af3be8568d880a69f1046c70ec"
server_key_version: "encryptionkey@0"
I20260812 04:01:57.296425 24104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:57.323118 24104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:57.325723 24104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:57.325852 24104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:57.330142 24104 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:36433
I20260812 04:01:57.330216 24177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:36433 every 8 connection(s)
I20260812 04:01:57.330579 24104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:57.331703 24178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:57.334424 24178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d: Bootstrap starting.
I20260812 04:01:57.335145 24178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:57.335497 24178 log.cc:824] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:57.336304 24178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d: No bootstrap required, opened a new log
I20260812 04:01:57.337047 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 24104
I20260812 04:01:57.337186 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:57.337412 31919 external_mini_cluster.cc:1428] Setting key 3593c0f3ad11e13018e01bf0a5835aec
I20260812 04:01:57.339381 24178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b996b1e9baa451f9ff7ced26b3d1c1d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 36433 } }
I20260812 04:01:57.339574 24178 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:57.339622 24178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b996b1e9baa451f9ff7ced26b3d1c1d, State: Initialized, Role: FOLLOWER
I20260812 04:01:57.339776 24178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b996b1e9baa451f9ff7ced26b3d1c1d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 36433 } }
I20260812 04:01:57.339872 24178 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:57.339919 24178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:57.339964 24178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:57.340584 24178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b996b1e9baa451f9ff7ced26b3d1c1d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 36433 } }
I20260812 04:01:57.340708 24178 leader_election.cc:302] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b996b1e9baa451f9ff7ced26b3d1c1d; no voters:
I20260812 04:01:57.340910 24178 leader_election.cc:288] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:57.340956 24183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:57.341146 24183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [term 1 LEADER]: Becoming Leader. State: Replica: 6b996b1e9baa451f9ff7ced26b3d1c1d, State: Running, Role: LEADER
I20260812 04:01:57.341240 24178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:57.341257 24183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b996b1e9baa451f9ff7ced26b3d1c1d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 36433 } }
I20260812 04:01:57.341583 24183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b996b1e9baa451f9ff7ced26b3d1c1d. Latest consensus state: current_term: 1 leader_uuid: "6b996b1e9baa451f9ff7ced26b3d1c1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b996b1e9baa451f9ff7ced26b3d1c1d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 36433 } } }
I20260812 04:01:57.341674 24183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:57.341595 24185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b996b1e9baa451f9ff7ced26b3d1c1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b996b1e9baa451f9ff7ced26b3d1c1d" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 36433 } } }
I20260812 04:01:57.341835 24185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d [sys.catalog]: This master's current role is: LEADER
W20260812 04:01:57.343181 24201 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:01:57.343456 24201 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:01:57.343554 24191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:57.343786 24191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:57.344303 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:57.337993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36421 (local address 127.31.43.254:36433)
0812 04:01:57.338198 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:57.338209 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:57.338247 (+ 38us) server_negotiation.cc:407] Connection header received
0812 04:01:57.339302 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:57.339310 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:57.339364 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:57.339553 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:57.339789 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:57.340821 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:57.341881 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:57.342369 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:57.342432 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:57.343413 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:57.343418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:57.343419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:57.343639 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:57.343683 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:57.343699 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:57.343786 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:57.344054 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 04:01:57.344085 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260812 04:01:57.345367 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:57.347185 24191 catalog_manager.cc:1347] Generated new cluster ID: 5059c6abdc784034932853cf2b93fb04
I20260812 04:01:57.347254 24191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:57.351424 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:57.347380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51011 (local address 127.31.43.254:36433)
0812 04:01:57.347423 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:57.347425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:57.347708 (+ 283us) server_negotiation.cc:407] Connection header received
0812 04:01:57.347761 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:57.347762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:57.347774 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:57.347836 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:57.348045 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:57.348812 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:57.349931 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:01:57.350269 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:01:57.350348 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:01:57.351148 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:57.351150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:57.351150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:57.351155 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:57.351175 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:57.351179 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:57.351241 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:57.351328 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 04:01:57.351342 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 04:01:57.364885 24191 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:57.365607 24191 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:57.371908 24191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b996b1e9baa451f9ff7ced26b3d1c1d: Generated new TSK 0
I20260812 04:01:57.372144 24191 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:57.379453 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 24104
W20260812 04:01:57.388196 24087 connection.cc:535] client connection to 127.31.43.254:36433 recv error: Network error: failed to read from TLS socket (remote: 127.31.43.254:36433): Connection reset by peer (error 104)
2026-08-12T04:01:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:57 dist-test-slave-8tdl krb5kdc[24214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:57 dist-test-slave-8tdl krb5kdc[24214](info): set up 2 sockets
Aug 12 04:01:57 dist-test-slave-8tdl krb5kdc[24214](info): commencing operation
krb5kdc: starting...
W20260812 04:01:58.504237 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:01:58 dist-test-slave-8tdl krb5kdc[24214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:01:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:01:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:01:58.638489 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40037
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:40037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:01:58.724790 24229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:01:58.725064 24229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:01:58.725116 24229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:01:58.726408 24229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:01:58.726486 24229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:01:58.726511 24229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:01:58.726533 24229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:01:58.726560 24229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:01:58.728245 24229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:40037
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:40037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:01:58.728581 24229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:01:58.728935 24229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:01:58.734277 24234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:58.734236 24236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:01:58.734261 24239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:01:58.734584 24229 server_base.cc:1034] running on GCE node
I20260812 04:01:58.734886 24229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:01:58.735214 24229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:01:58.736407 24229 hybrid_clock.cc:648] HybridClock initialized: now 1786507318736391 us; error 43 us; skew 500 ppm
Aug 12 04:01:58 dist-test-slave-8tdl krb5kdc[24214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507318, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:01:58.743013 24229 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:01:58.743569 24229 webserver.cc:460] Webserver started at http://127.31.43.254:41647/ using document root <none> and password file <none>
I20260812 04:01:58.743824 24229 fs_manager.cc:359] Metadata directory not provided
I20260812 04:01:58.743877 24229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:01:58.744019 24229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:01:58.745301 24229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "811a89a490504671bdb8fb5a81b193fd"
format_stamp: "Formatted at 2026-08-12 04:01:58 on dist-test-slave-8tdl"
server_key: "b1031a418a005a662ef506837a29821a"
server_key_iv: "da9cfca04ecae6ad7b6af72b6bb4684b"
server_key_version: "encryptionkey@0"
I20260812 04:01:58.745720 24229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "811a89a490504671bdb8fb5a81b193fd"
format_stamp: "Formatted at 2026-08-12 04:01:58 on dist-test-slave-8tdl"
server_key: "b1031a418a005a662ef506837a29821a"
server_key_iv: "da9cfca04ecae6ad7b6af72b6bb4684b"
server_key_version: "encryptionkey@0"
I20260812 04:01:58.747771 24229 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 04:01:58.748793 24248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:01:58.749135 24229 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:01:58.749248 24229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "811a89a490504671bdb8fb5a81b193fd"
format_stamp: "Formatted at 2026-08-12 04:01:58 on dist-test-slave-8tdl"
server_key: "b1031a418a005a662ef506837a29821a"
server_key_iv: "da9cfca04ecae6ad7b6af72b6bb4684b"
server_key_version: "encryptionkey@0"
I20260812 04:01:58.749331 24229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:01:58.752882 24229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:01:58.755342 24229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:01:58.755527 24229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:01:58.759771 24229 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:40037
I20260812 04:01:58.759824 24300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:40037 every 8 connection(s)
I20260812 04:01:58.760178 24229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:01:58.761389 24301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:01:58.763994 24301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd: Bootstrap starting.
I20260812 04:01:58.764637 24301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd: Neither blocks nor log segments found. Creating new log.
I20260812 04:01:58.765021 24301 log.cc:824] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd: Log is configured to *not* fsync() on all Append() calls
I20260812 04:01:58.765039 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 24229
I20260812 04:01:58.765164 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:01:58.765367 31919 external_mini_cluster.cc:1428] Setting key 9b29306ba02a704c04df2ca95003a830
I20260812 04:01:58.765977 24301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd: No bootstrap required, opened a new log
I20260812 04:01:58.768401 24301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811a89a490504671bdb8fb5a81b193fd" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40037 } }
I20260812 04:01:58.768636 24301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:01:58.768689 24301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 811a89a490504671bdb8fb5a81b193fd, State: Initialized, Role: FOLLOWER
I20260812 04:01:58.768819 24301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811a89a490504671bdb8fb5a81b193fd" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40037 } }
I20260812 04:01:58.768925 24301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:01:58.769043 24301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:01:58.768906 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:58.765931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56545 (local address 127.31.43.254:40037)
0812 04:01:58.766159 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:58.766167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:58.766194 (+ 27us) server_negotiation.cc:407] Connection header received
0812 04:01:58.767306 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:58.767315 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:58.767354 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:58.767484 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:58.768188 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:58.768191 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:58.768191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:58.768195 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:58.768237 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:58.768248 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:58.768310 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:58.768411 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 04:01:58.768479 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":101,"threads_started":1}
I20260812 04:01:58.769107 24301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:01:58.769886 24301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811a89a490504671bdb8fb5a81b193fd" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40037 } }
I20260812 04:01:58.770004 24301 leader_election.cc:302] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 811a89a490504671bdb8fb5a81b193fd; no voters:
I20260812 04:01:58.770108 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:01:58.770169 24301 leader_election.cc:288] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:01:58.770229 24306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:01:58.770395 24306 raft_consensus.cc:686] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [term 1 LEADER]: Becoming Leader. State: Replica: 811a89a490504671bdb8fb5a81b193fd, State: Running, Role: LEADER
I20260812 04:01:58.770404 24301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:01:58.770500 24306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811a89a490504671bdb8fb5a81b193fd" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40037 } }
I20260812 04:01:58.770874 24306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 811a89a490504671bdb8fb5a81b193fd. Latest consensus state: current_term: 1 leader_uuid: "811a89a490504671bdb8fb5a81b193fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811a89a490504671bdb8fb5a81b193fd" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40037 } } }
I20260812 04:01:58.770994 24306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:58.770888 24307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "811a89a490504671bdb8fb5a81b193fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811a89a490504671bdb8fb5a81b193fd" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 40037 } } }
I20260812 04:01:58.771140 24307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd [sys.catalog]: This master's current role is: LEADER
I20260812 04:01:58.771234 24313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:01:58.771873 24313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:01:58.773049 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:01:58.771725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51321 (local address 127.31.43.254:40037)
0812 04:01:58.771767 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 04:01:58.771769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:01:58.772067 (+ 298us) server_negotiation.cc:407] Connection header received
0812 04:01:58.772166 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:01:58.772167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:01:58.772173 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:01:58.772251 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:01:58.772796 (+ 545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:01:58.772797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:01:58.772797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:01:58.772802 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:01:58.772818 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:01:58.772824 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:01:58.772884 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 04:01:58.772944 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 04:01:58.772959 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 04:01:58.774595 24313 catalog_manager.cc:1347] Generated new cluster ID: 0bb5c60dfb3748e98f92ce7535cc5b3a
I20260812 04:01:58.774652 24313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:01:58.783334 24313 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:01:58.783526 24313 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:01:58.796888 24313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 811a89a490504671bdb8fb5a81b193fd: Generated new TSK 0
I20260812 04:01:58.797205 24313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:01:58.804051 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 24229
2026-08-12T04:01:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:01:58 dist-test-slave-8tdl krb5kdc[24336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:01:58 dist-test-slave-8tdl krb5kdc[24336](info): set up 2 sockets
Aug 12 04:01:58 dist-test-slave-8tdl krb5kdc[24336](info): commencing operation
krb5kdc: starting...
W20260812 04:01:59.928268 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:02:00 dist-test-slave-8tdl krb5kdc[24336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:02:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:02:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:02:00.070189 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:41997
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:45025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:41997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:02:00.157616 24349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:02:00.157884 24349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:02:00.157927 24349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:02:00.159180 24349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:02:00.159257 24349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:02:00.159282 24349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:02:00.159303 24349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:02:00.159332 24349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:02:00.160914 24349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:45025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:41997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:41997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:02:00.161229 24349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:02:00.161481 24349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:02:00.166957 24354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:00.166965 24355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:00.166939 24358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:02:00.166939 24349 server_base.cc:1034] running on GCE node
I20260812 04:02:00.167388 24349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:02:00.167655 24349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:02:00.168825 24349 hybrid_clock.cc:648] HybridClock initialized: now 1786507320168799 us; error 36 us; skew 500 ppm
Aug 12 04:02:00 dist-test-slave-8tdl krb5kdc[24336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507320, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:02:00.171650 24349 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:02:00.172216 24349 webserver.cc:460] Webserver started at http://127.31.43.254:41469/ using document root <none> and password file <none>
I20260812 04:02:00.172485 24349 fs_manager.cc:359] Metadata directory not provided
I20260812 04:02:00.172556 24349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:02:00.172705 24349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:02:00.173836 24349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "6be4527ca33944479161cbc3fd3ad244"
format_stamp: "Formatted at 2026-08-12 04:02:00 on dist-test-slave-8tdl"
server_key: "493dc66697a0343448bd5c731e4c759f"
server_key_iv: "f7959545064dc76b2e5351048ec5ba77"
server_key_version: "encryptionkey@0"
I20260812 04:02:00.174211 24349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "6be4527ca33944479161cbc3fd3ad244"
format_stamp: "Formatted at 2026-08-12 04:02:00 on dist-test-slave-8tdl"
server_key: "493dc66697a0343448bd5c731e4c759f"
server_key_iv: "f7959545064dc76b2e5351048ec5ba77"
server_key_version: "encryptionkey@0"
I20260812 04:02:00.175769 24349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 04:02:00.176618 24365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:02:00.176837 24349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:02:00.176959 24349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "6be4527ca33944479161cbc3fd3ad244"
format_stamp: "Formatted at 2026-08-12 04:02:00 on dist-test-slave-8tdl"
server_key: "493dc66697a0343448bd5c731e4c759f"
server_key_iv: "f7959545064dc76b2e5351048ec5ba77"
server_key_version: "encryptionkey@0"
I20260812 04:02:00.177040 24349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:02:00.189271 24349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:02:00.191627 24349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:02:00.191807 24349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:02:00.196313 24349 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:41997
I20260812 04:02:00.196345 24424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:41997 every 8 connection(s)
I20260812 04:02:00.196738 24349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:02:00.197724 24425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:02:00.200436 24425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244: Bootstrap starting.
I20260812 04:02:00.201206 24425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244: Neither blocks nor log segments found. Creating new log.
I20260812 04:02:00.201615 24425 log.cc:824] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244: Log is configured to *not* fsync() on all Append() calls
I20260812 04:02:00.202515 24425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244: No bootstrap required, opened a new log
I20260812 04:02:00.204551 24425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be4527ca33944479161cbc3fd3ad244" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41997 } }
I20260812 04:02:00.204744 24425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:02:00.204798 24425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6be4527ca33944479161cbc3fd3ad244, State: Initialized, Role: FOLLOWER
I20260812 04:02:00.205009 24425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be4527ca33944479161cbc3fd3ad244" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41997 } }
I20260812 04:02:00.205125 24425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:02:00.205178 24425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:02:00.205240 24425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:02:00.206255 24425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be4527ca33944479161cbc3fd3ad244" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41997 } }
I20260812 04:02:00.206418 24425 leader_election.cc:302] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6be4527ca33944479161cbc3fd3ad244; no voters:
I20260812 04:02:00.206624 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 24349
I20260812 04:02:00.206655 24425 leader_election.cc:288] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:02:00.206705 24428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:02:00.206789 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:02:00.206849 24428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [term 1 LEADER]: Becoming Leader. State: Replica: 6be4527ca33944479161cbc3fd3ad244, State: Running, Role: LEADER
I20260812 04:02:00.206990 24425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:02:00.206946 24428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be4527ca33944479161cbc3fd3ad244" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41997 } }
I20260812 04:02:00.207033 31919 external_mini_cluster.cc:1428] Setting key 6317ec4cbd8a1e1e6297765934665fb5
I20260812 04:02:00.207345 24428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6be4527ca33944479161cbc3fd3ad244. Latest consensus state: current_term: 1 leader_uuid: "6be4527ca33944479161cbc3fd3ad244" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be4527ca33944479161cbc3fd3ad244" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41997 } } }
I20260812 04:02:00.207363 24429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6be4527ca33944479161cbc3fd3ad244" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be4527ca33944479161cbc3fd3ad244" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 41997 } } }
I20260812 04:02:00.207420 24428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [sys.catalog]: This master's current role is: LEADER
I20260812 04:02:00.207479 24429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244 [sys.catalog]: This master's current role is: LEADER
I20260812 04:02:00.207697 24433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:02:00.208348 24433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:02:00.210254 24433 catalog_manager.cc:1347] Generated new cluster ID: c6b2b0799fe1424d8bf139c1f95198f7
I20260812 04:02:00.210361 24433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 04:02:00 dist-test-slave-8tdl krb5kdc[24336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:02:00.221592 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:00.207803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47527 (local address 127.31.43.254:41997)
0812 04:02:00.208351 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:00.208362 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:00.208398 (+ 36us) server_negotiation.cc:407] Connection header received
0812 04:02:00.208537 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:00.208546 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:00.208610 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:00.208794 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:00.209560 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:00.211055 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:00.212297 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:00.212927 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:00.213059 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:00.216298 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:00.216304 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:00.216305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:00.216322 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:00.219537 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:00.220001 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:00.220005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:00.220006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:00.220230 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:00.220461 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:00.220462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:00.220462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:00.220850 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:00.221054 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:00.221133 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 04:02:00.221303 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":81,"threads_started":1}
I20260812 04:02:00.223132 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:02:00.227138 24433 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:02:00.227628 24433 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:02:00.233187 24433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6be4527ca33944479161cbc3fd3ad244: Generated new TSK 0
I20260812 04:02:00.233393 24433 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:02:00.233698 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:00.224660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59225 (local address 127.31.43.254:41997)
0812 04:02:00.224707 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:00.224709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:00.224880 (+ 171us) server_negotiation.cc:407] Connection header received
0812 04:02:00.224948 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:00.224949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:00.224970 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:00.225034 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:00.225709 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:00.226772 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:00.228134 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:00.228497 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:00.228564 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:00.230273 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:00.230275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:00.230275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:00.230283 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:00.232020 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:00.232474 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:00.232475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:00.232476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:00.232537 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:00.232745 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:00.232746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:00.232746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:00.232881 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:00.233002 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:00.233469 (+ 467us) server_negotiation.cc:299] Negotiation successful
0812 04:02:00.233582 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 04:02:00.235961 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 24349
2026-08-12T04:02:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:02:00 dist-test-slave-8tdl krb5kdc[24458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:02:00 dist-test-slave-8tdl krb5kdc[24458](info): set up 2 sockets
Aug 12 04:02:00 dist-test-slave-8tdl krb5kdc[24458](info): commencing operation
krb5kdc: starting...
W20260812 04:02:01.361584 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:02:01 dist-test-slave-8tdl krb5kdc[24458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:02:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:02:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:02:01.498811 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:39625
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:46115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:39625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:02:01.589766 24471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:02:01.590011 24471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:02:01.590037 24471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:02:01.591215 24471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:02:01.591259 24471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:02:01.591274 24471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:02:01.591286 24471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:02:01.591298 24471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:02:01.592782 24471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:46115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:39625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:39625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:02:01.593004 24471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:02:01.593200 24471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:02:01.598376 24476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:01.598534 24478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:01.598534 24480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:02:01.598737 24471 server_base.cc:1034] running on GCE node
I20260812 04:02:01.599035 24471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:02:01.599303 24471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:02:01.600458 24471 hybrid_clock.cc:648] HybridClock initialized: now 1786507321600444 us; error 30 us; skew 500 ppm
Aug 12 04:02:01 dist-test-slave-8tdl krb5kdc[24458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507321, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:02:01.603101 24471 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:02:01.603570 24471 webserver.cc:460] Webserver started at http://127.31.43.254:44787/ using document root <none> and password file <none>
I20260812 04:02:01.603801 24471 fs_manager.cc:359] Metadata directory not provided
I20260812 04:02:01.603866 24471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:02:01.604019 24471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:02:01.605106 24471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "4cdec88bf1234d96ad4cd34ca4b9fd95"
format_stamp: "Formatted at 2026-08-12 04:02:01 on dist-test-slave-8tdl"
server_key: "34dbbd238a4cb2f6933b75bb8d4cc78c"
server_key_iv: "b05fdb9d5e7948b6a2ce964cce4b5ecb"
server_key_version: "encryptionkey@0"
I20260812 04:02:01.605484 24471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "4cdec88bf1234d96ad4cd34ca4b9fd95"
format_stamp: "Formatted at 2026-08-12 04:02:01 on dist-test-slave-8tdl"
server_key: "34dbbd238a4cb2f6933b75bb8d4cc78c"
server_key_iv: "b05fdb9d5e7948b6a2ce964cce4b5ecb"
server_key_version: "encryptionkey@0"
I20260812 04:02:01.607087 24471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 04:02:01.607985 24487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:02:01.608198 24471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:02:01.608306 24471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "4cdec88bf1234d96ad4cd34ca4b9fd95"
format_stamp: "Formatted at 2026-08-12 04:02:01 on dist-test-slave-8tdl"
server_key: "34dbbd238a4cb2f6933b75bb8d4cc78c"
server_key_iv: "b05fdb9d5e7948b6a2ce964cce4b5ecb"
server_key_version: "encryptionkey@0"
I20260812 04:02:01.608392 24471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:02:01.625734 24471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:02:01.628131 24471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:02:01.628304 24471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:02:01.632732 24471 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:39625
I20260812 04:02:01.632774 24548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:39625 every 8 connection(s)
I20260812 04:02:01.633176 24471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:02:01.634130 24549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:02:01.635299 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 24471
I20260812 04:02:01.635433 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:02:01.635618 31919 external_mini_cluster.cc:1428] Setting key 1ef19709a06698dcb9115f91a766eda6
I20260812 04:02:01.637049 24549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95: Bootstrap starting.
I20260812 04:02:01.637799 24549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95: Neither blocks nor log segments found. Creating new log.
I20260812 04:02:01.638108 24549 log.cc:824] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95: Log is configured to *not* fsync() on all Append() calls
I20260812 04:02:01.638962 24549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95: No bootstrap required, opened a new log
I20260812 04:02:01.641333 24549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdec88bf1234d96ad4cd34ca4b9fd95" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 39625 } }
I20260812 04:02:01.641515 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:01.636149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33921 (local address 127.31.43.254:39625)
0812 04:02:01.636324 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:01.636332 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:01.636357 (+ 25us) server_negotiation.cc:407] Connection header received
0812 04:02:01.636787 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:01.636793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:01.636841 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:01.636955 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:01.637255 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:01.638291 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:01.639450 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:01.639875 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:01.639991 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:01.640714 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:01.640717 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:01.640718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:02:01.640881 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:01.640914 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:02:01.640929 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:01.640994 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:01.641118 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 04:02:01.641307 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260812 04:02:01.641525 24549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:02:01.641683 24549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cdec88bf1234d96ad4cd34ca4b9fd95, State: Initialized, Role: FOLLOWER
I20260812 04:02:01.641986 24549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdec88bf1234d96ad4cd34ca4b9fd95" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 39625 } }
I20260812 04:02:01.642104 24549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:02:01.642167 24549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:02:01.642239 24549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:02:01.642562 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:02:01.642947 24549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdec88bf1234d96ad4cd34ca4b9fd95" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 39625 } }
I20260812 04:02:01.643072 24549 leader_election.cc:302] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cdec88bf1234d96ad4cd34ca4b9fd95; no voters:
I20260812 04:02:01.643266 24549 leader_election.cc:288] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:02:01.643289 24554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:02:01.643410 24554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [term 1 LEADER]: Becoming Leader. State: Replica: 4cdec88bf1234d96ad4cd34ca4b9fd95, State: Running, Role: LEADER
I20260812 04:02:01.643551 24549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:02:01.643567 24554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdec88bf1234d96ad4cd34ca4b9fd95" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 39625 } }
I20260812 04:02:01.643965 24554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cdec88bf1234d96ad4cd34ca4b9fd95. Latest consensus state: current_term: 1 leader_uuid: "4cdec88bf1234d96ad4cd34ca4b9fd95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdec88bf1234d96ad4cd34ca4b9fd95" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 39625 } } }
I20260812 04:02:01.644080 24555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cdec88bf1234d96ad4cd34ca4b9fd95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdec88bf1234d96ad4cd34ca4b9fd95" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 39625 } } }
I20260812 04:02:01.644111 24554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [sys.catalog]: This master's current role is: LEADER
I20260812 04:02:01.644181 24555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95 [sys.catalog]: This master's current role is: LEADER
I20260812 04:02:01.644388 24565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:02:01.645084 24565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:02:01.646917 24565 catalog_manager.cc:1347] Generated new cluster ID: 4e589b92ef65410285f4439711fa8acc
I20260812 04:02:01.647027 24565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:02:01.648991 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:01.645351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53141 (local address 127.31.43.254:39625)
0812 04:02:01.645388 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:01.645390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:01.645669 (+ 279us) server_negotiation.cc:407] Connection header received
0812 04:02:01.645764 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:01.645765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:01.645774 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:01.645802 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:01.646001 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:01.646664 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:01.647682 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:01.647980 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:01.648053 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:01.648749 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:01.648751 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:01.648751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:02:01.648756 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:01.648773 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:02:01.648778 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:01.648832 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:01.648892 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 04:02:01.648916 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 04:02:01.650820 24565 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:02:01.651335 24565 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:02:01.655390 24565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cdec88bf1234d96ad4cd34ca4b9fd95: Generated new TSK 0
I20260812 04:02:01.655563 24565 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:02:01.661103 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 24471
2026-08-12T04:02:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:02:01 dist-test-slave-8tdl krb5kdc[24583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:02:01 dist-test-slave-8tdl krb5kdc[24583](info): set up 2 sockets
Aug 12 04:02:01 dist-test-slave-8tdl krb5kdc[24583](info): commencing operation
krb5kdc: starting...
W20260812 04:02:02.789822 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:02:02 dist-test-slave-8tdl krb5kdc[24583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:02:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:02:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:02:02.925966 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:43167
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:32931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:43167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:02:03.014521 24596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:02:03.014762 24596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:02:03.014787 24596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:02:03.015933 24596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:02:03.015982 24596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:02:03.015997 24596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:02:03.016009 24596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:02:03.016021 24596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:02:03.017549 24596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:32931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:43167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:43167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:02:03.017771 24596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:02:03.017954 24596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:02:03.023152 24602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:03.023160 24604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:02:03.023228 24596 server_base.cc:1034] running on GCE node
W20260812 04:02:03.023181 24601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:02:03.023723 24596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:02:03.024050 24596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:02:03.025228 24596 hybrid_clock.cc:648] HybridClock initialized: now 1786507323025186 us; error 69 us; skew 500 ppm
Aug 12 04:02:03 dist-test-slave-8tdl krb5kdc[24583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507323, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:02:03.028004 24596 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:02:03.028497 24596 webserver.cc:460] Webserver started at http://127.31.43.254:38085/ using document root <none> and password file <none>
I20260812 04:02:03.028735 24596 fs_manager.cc:359] Metadata directory not provided
I20260812 04:02:03.028802 24596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:02:03.028963 24596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:02:03.030118 24596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "cd51c8154cf54a8695bcf2cdd5bbe1d2"
format_stamp: "Formatted at 2026-08-12 04:02:03 on dist-test-slave-8tdl"
server_key: "9e3741e5875ae34adc0b91b6b41a152b"
server_key_iv: "c51b8c6d671c89479bcd148bc3416cbd"
server_key_version: "encryptionkey@0"
I20260812 04:02:03.030498 24596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "cd51c8154cf54a8695bcf2cdd5bbe1d2"
format_stamp: "Formatted at 2026-08-12 04:02:03 on dist-test-slave-8tdl"
server_key: "9e3741e5875ae34adc0b91b6b41a152b"
server_key_iv: "c51b8c6d671c89479bcd148bc3416cbd"
server_key_version: "encryptionkey@0"
I20260812 04:02:03.032064 24596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 04:02:03.033021 24615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:02:03.033247 24596 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 04:02:03.033370 24596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "cd51c8154cf54a8695bcf2cdd5bbe1d2"
format_stamp: "Formatted at 2026-08-12 04:02:03 on dist-test-slave-8tdl"
server_key: "9e3741e5875ae34adc0b91b6b41a152b"
server_key_iv: "c51b8c6d671c89479bcd148bc3416cbd"
server_key_version: "encryptionkey@0"
I20260812 04:02:03.033458 24596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:02:03.037024 24596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:02:03.039191 24596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:02:03.039354 24596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:02:03.043637 24596 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:43167
I20260812 04:02:03.043689 24672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:43167 every 8 connection(s)
I20260812 04:02:03.044054 24596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:02:03.045122 24673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:02:03.047780 24673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2: Bootstrap starting.
I20260812 04:02:03.048339 24673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2: Neither blocks nor log segments found. Creating new log.
I20260812 04:02:03.048584 24673 log.cc:824] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2: Log is configured to *not* fsync() on all Append() calls
I20260812 04:02:03.049355 24673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2: No bootstrap required, opened a new log
I20260812 04:02:03.051019 24673 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd51c8154cf54a8695bcf2cdd5bbe1d2" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43167 } }
I20260812 04:02:03.051203 24673 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 04:02:03.051225 24673 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd51c8154cf54a8695bcf2cdd5bbe1d2, State: Initialized, Role: FOLLOWER
I20260812 04:02:03.051357 24673 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd51c8154cf54a8695bcf2cdd5bbe1d2" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43167 } }
I20260812 04:02:03.051443 24673 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 04:02:03.051467 24673 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 04:02:03.051504 24673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:02:03.052037 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 24596
I20260812 04:02:03.052183 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:02:03.052371 31919 external_mini_cluster.cc:1428] Setting key b41d6bcfad70c960f621bb9c9e303f01
I20260812 04:02:03.052462 24673 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd51c8154cf54a8695bcf2cdd5bbe1d2" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43167 } }
I20260812 04:02:03.052589 24673 leader_election.cc:302] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd51c8154cf54a8695bcf2cdd5bbe1d2; no voters:
I20260812 04:02:03.052783 24673 leader_election.cc:288] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 04:02:03.052837 24676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:02:03.053164 24676 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [term 1 LEADER]: Becoming Leader. State: Replica: cd51c8154cf54a8695bcf2cdd5bbe1d2, State: Running, Role: LEADER
I20260812 04:02:03.053247 24673 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:02:03.053268 24676 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd51c8154cf54a8695bcf2cdd5bbe1d2" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43167 } }
I20260812 04:02:03.053640 24676 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd51c8154cf54a8695bcf2cdd5bbe1d2. Latest consensus state: current_term: 1 leader_uuid: "cd51c8154cf54a8695bcf2cdd5bbe1d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd51c8154cf54a8695bcf2cdd5bbe1d2" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43167 } } }
I20260812 04:02:03.053782 24676 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [sys.catalog]: This master's current role is: LEADER
I20260812 04:02:03.053900 24680 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd51c8154cf54a8695bcf2cdd5bbe1d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd51c8154cf54a8695bcf2cdd5bbe1d2" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 43167 } } }
I20260812 04:02:03.054113 24680 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2 [sys.catalog]: This master's current role is: LEADER
I20260812 04:02:03.054061 24684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:02:03.055022 24679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:03.052945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38125 (local address 127.31.43.254:43167)
0812 04:02:03.053145 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:03.053154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:03.053211 (+ 57us) server_negotiation.cc:407] Connection header received
0812 04:02:03.053333 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:03.053340 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:03.053372 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:03.053520 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:03.054288 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:03.054292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:03.054292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:02:03.054298 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:03.054342 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:02:03.054358 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:03.054401 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:03.054762 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 04:02:03.054810 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
I20260812 04:02:03.055297 24684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:02:03.056174 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:02:03.057629 24684 catalog_manager.cc:1347] Generated new cluster ID: 43ad061291c74e2ba71ad72f0c154c47
I20260812 04:02:03.057691 24684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:02:03.058996 24679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:03.057773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51685 (local address 127.31.43.254:43167)
0812 04:02:03.057813 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:03.057815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:03.057999 (+ 184us) server_negotiation.cc:407] Connection header received
0812 04:02:03.058070 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:03.058071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:03.058076 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:03.058133 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:03.058786 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:03.058787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:03.058787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 04:02:03.058791 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:03.058805 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 04:02:03.058809 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:03.058854 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:03.058918 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 04:02:03.058930 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 04:02:03.064332 24684 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:02:03.064525 24684 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:02:03.069518 24684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd51c8154cf54a8695bcf2cdd5bbe1d2: Generated new TSK 0
I20260812 04:02:03.069702 24684 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 04:02:03.079651 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 24596
2026-08-12T04:02:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1418 ms)
[----------] 18 tests from AuthTokenIssuingTest (26055 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:02:03 dist-test-slave-8tdl krb5kdc[24709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:02:03 dist-test-slave-8tdl krb5kdc[24709](info): set up 2 sockets
Aug 12 04:02:03 dist-test-slave-8tdl krb5kdc[24709](info): commencing operation
krb5kdc: starting...
W20260812 04:02:04.215909 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:02:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:02:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:02:04.349716 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:33033
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:43393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:33033,127.31.43.253:44103,127.31.43.252:34591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:02:04.438427 24722 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 04:02:04.438807 24722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:02:04.438843 24722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:02:04.438891 24722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:02:04.440161 24722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:02:04.440248 24722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:02:04.440270 24722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:02:04.440292 24722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:02:04.440320 24722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:02:04.442034 24722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:43393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:33033,127.31.43.253:44103,127.31.43.252:34591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:33033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:02:04.442366 24722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:02:04.442628 24722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:02:04.448496 24727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:04.448493 24728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:02:04.448673 24722 server_base.cc:1034] running on GCE node
W20260812 04:02:04.448505 24730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:02:04.448942 24722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:02:04.449278 24722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:02:04.450448 24722 hybrid_clock.cc:648] HybridClock initialized: now 1786507324450433 us; error 33 us; skew 500 ppm
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:02:04.460412 24722 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:02:04.461001 24722 webserver.cc:460] Webserver started at http://127.31.43.254:41663/ using document root <none> and password file <none>
I20260812 04:02:04.461287 24722 fs_manager.cc:359] Metadata directory not provided
I20260812 04:02:04.461369 24722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:02:04.461548 24722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:02:04.462883 24722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "d31720c9355f4396837017449d0e4c68"
format_stamp: "Formatted at 2026-08-12 04:02:04 on dist-test-slave-8tdl"
server_key: "debf6911178282879cb3a207ad913a85"
server_key_iv: "865dd06c4fb3240de5ae8bb471d94b1f"
server_key_version: "encryptionkey@0"
I20260812 04:02:04.463305 24722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "d31720c9355f4396837017449d0e4c68"
format_stamp: "Formatted at 2026-08-12 04:02:04 on dist-test-slave-8tdl"
server_key: "debf6911178282879cb3a207ad913a85"
server_key_iv: "865dd06c4fb3240de5ae8bb471d94b1f"
server_key_version: "encryptionkey@0"
I20260812 04:02:04.464879 24722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 04:02:04.465783 24738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:02:04.465974 24722 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:02:04.466099 24722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "d31720c9355f4396837017449d0e4c68"
format_stamp: "Formatted at 2026-08-12 04:02:04 on dist-test-slave-8tdl"
server_key: "debf6911178282879cb3a207ad913a85"
server_key_iv: "865dd06c4fb3240de5ae8bb471d94b1f"
server_key_version: "encryptionkey@0"
I20260812 04:02:04.466198 24722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:02:04.480687 24722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:02:04.483407 24722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:02:04.483636 24722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:02:04.488850 24722 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:33033
I20260812 04:02:04.488884 24795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:33033 every 8 connection(s)
I20260812 04:02:04.489336 24722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:02:04.490420 24796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:02:04.492219 24796 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:02:04.496260 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 24722
I20260812 04:02:04.496397 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:02:04.496622 31919 external_mini_cluster.cc:1428] Setting key f495433b3da8a8adb699882d87bb10af
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:02:04.502745 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.493073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35778 (local address 127.31.43.254:33033)
0812 04:02:04.493229 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.493239 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.493376 (+ 137us) server_negotiation.cc:407] Connection header received
0812 04:02:04.493956 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.493958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.493979 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.494087 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.494651 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.495543 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.496424 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.496847 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.496978 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.499236 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.499245 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.499245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.499254 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.500922 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.501440 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.501444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.501445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.501536 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.501830 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.501831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.501831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.502046 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.502143 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.502373 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.502503 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260812 04:02:04.502832 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.492943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33033 (local address 127.0.0.1:35778)
0812 04:02:04.493216 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.493271 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:04.493790 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:04.494093 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:04.494095 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:04.494382 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:04.494541 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.494546 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.495657 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.495658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.496314 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.496316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.496966 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.496967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.497047 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.497642 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:04.497651 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:04.499141 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:04.501019 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.501029 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.501032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.501339 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.501613 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.501614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.501615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.501760 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.502130 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:04.502137 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:04.502224 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 04:02:04.502340 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 04:02:04.502598 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":56,"thread_start_us":147,"threads_started":1}
I20260812 04:02:04.503749 24796 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:02:04.504142 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.504019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:44103 (local address 127.0.0.1:51132)
0812 04:02:04.504071 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.504120 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:44103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 04:02:04.504321 24739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.43.253:44103: connect: Connection refused (error 111)
I20260812 04:02:04.504814 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.504729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:44103 (local address 127.0.0.1:51144)
0812 04:02:04.504772 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.504797 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:44103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 04:02:04.504940 24796 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.253:44103: Network error: Client connection negotiation failed: client connection to 127.31.43.253:44103: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.31.43.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.253.keytab.
Entry for principal kudu/127.31.43.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.253.keytab.
I20260812 04:02:04.526952 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.253:44103
--webserver_interface=127.31.43.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.253.keytab
--principal=kudu/127.31.43.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:43393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:33033,127.31.43.253:44103,127.31.43.252:34591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:02:04.554194 24796 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } attempt: 1
I20260812 04:02:04.554895 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.554650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:44103 (local address 127.0.0.1:51156)
0812 04:02:04.554791 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.554840 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:44103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260812 04:02:04.555382 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.555238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:44103 (local address 127.0.0.1:51164)
0812 04:02:04.555321 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.555355 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:44103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 04:02:04.555539 24796 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.253:44103: Network error: Client connection negotiation failed: client connection to 127.31.43.253:44103: connect: Connection refused (error 111)
W20260812 04:02:04.615022 24804 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 04:02:04.615267 24804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:02:04.615286 24804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:02:04.615307 24804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:02:04.616598 24804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:02:04.616654 24804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:02:04.616669 24804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:02:04.616683 24804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:02:04.616695 24804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:02:04.618251 24804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:43393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:33033,127.31.43.253:44103,127.31.43.252:34591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.253.keytab
--principal=kudu/127.31.43.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.253:44103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.43.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:02:04.618479 24804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:02:04.618666 24804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 04:02:04.623724 24796 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } attempt: 2
W20260812 04:02:04.623858 24809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:04.623854 24810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:04.623888 24813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:02:04.624141 24804 server_base.cc:1034] running on GCE node
I20260812 04:02:04.624300 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.624162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:44103 (local address 127.0.0.1:51170)
0812 04:02:04.624218 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.624258 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:44103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 04:02:04.624513 24804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:02:04.624732 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.624640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:44103 (local address 127.0.0.1:51184)
0812 04:02:04.624674 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.624703 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:44103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260812 04:02:04.624820 24796 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.253:44103: Network error: Client connection negotiation failed: client connection to 127.31.43.253:44103: connect: Connection refused (error 111)
I20260812 04:02:04.624914 24804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:02:04.626086 24804 hybrid_clock.cc:648] HybridClock initialized: now 1786507324626072 us; error 30 us; skew 500 ppm
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:02:04.628942 24804 init.cc:377] Logged in from keytab as kudu/127.31.43.253@KRBTEST.COM (short username kudu)
I20260812 04:02:04.629432 24804 webserver.cc:460] Webserver started at http://127.31.43.253:38307/ using document root <none> and password file <none>
I20260812 04:02:04.629691 24804 fs_manager.cc:359] Metadata directory not provided
I20260812 04:02:04.629763 24804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:02:04.629920 24804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:02:04.631165 24804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/data/instance:
uuid: "6ebdb9bc2519475fbbd4a61360377229"
format_stamp: "Formatted at 2026-08-12 04:02:04 on dist-test-slave-8tdl"
server_key: "8155bcf3d7bf61a567569722435ccf4a"
server_key_iv: "0bf1b7e5e2884d6c01ac5c7c7c6b9609"
server_key_version: "encryptionkey@0"
I20260812 04:02:04.631564 24804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/wal/instance:
uuid: "6ebdb9bc2519475fbbd4a61360377229"
format_stamp: "Formatted at 2026-08-12 04:02:04 on dist-test-slave-8tdl"
server_key: "8155bcf3d7bf61a567569722435ccf4a"
server_key_iv: "0bf1b7e5e2884d6c01ac5c7c7c6b9609"
server_key_version: "encryptionkey@0"
I20260812 04:02:04.633068 24804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 04:02:04.633909 24820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:02:04.634100 24804 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:02:04.634213 24804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/wal
uuid: "6ebdb9bc2519475fbbd4a61360377229"
format_stamp: "Formatted at 2026-08-12 04:02:04 on dist-test-slave-8tdl"
server_key: "8155bcf3d7bf61a567569722435ccf4a"
server_key_iv: "0bf1b7e5e2884d6c01ac5c7c7c6b9609"
server_key_version: "encryptionkey@0"
I20260812 04:02:04.634294 24804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:02:04.643501 24804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:02:04.646056 24804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:02:04.646247 24804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:02:04.650909 24804 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.253:44103
I20260812 04:02:04.650944 24877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.253:44103 every 8 connection(s)
I20260812 04:02:04.651335 24804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
I20260812 04:02:04.652372 24878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:02:04.653067 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 24804
I20260812 04:02:04.653182 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-1/wal/instance
I20260812 04:02:04.653368 31919 external_mini_cluster.cc:1428] Setting key ab7f96d9fd954b8f4d7cbd086976e560
I20260812 04:02:04.654960 24878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.253@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.43.252@KRBTEST.COM; defaulting to no policy
I20260812 04:02:04.665645 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.655905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35794 (local address 127.31.43.254:33033)
0812 04:02:04.655979 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.655982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.656204 (+ 222us) server_negotiation.cc:407] Connection header received
0812 04:02:04.656684 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.656686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.656702 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.656766 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.657414 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.657918 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.659645 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.659852 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.659910 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.662501 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.662502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.662502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.662514 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.664096 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.664570 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.664572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.664573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.664643 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.664940 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.664941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.664942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.665071 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.665156 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.665446 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.665540 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 04:02:04.665866 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.655833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33033 (local address 127.0.0.1:35794)
0812 04:02:04.656061 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.656106 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:04.656567 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:04.656843 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:04.656846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:04.657109 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:04.657308 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.657312 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.658068 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.658069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.659541 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.659545 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.659959 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.659960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.660158 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.660736 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:04.660747 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:04.662394 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:04.664226 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.664235 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.664238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.664487 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.664721 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.664722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.664722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.664862 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.665147 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:04.665155 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:04.665308 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 04:02:04.665426 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 04:02:04.665616 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.252@KRBTEST.COM" created.
I20260812 04:02:04.666464 24878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.253@KRBTEST.COM for kudu/127.31.43.253@KRBTEST.COM
I20260812 04:02:04.674639 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.666837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:44103 (local address 127.0.0.1:51192)
0812 04:02:04.666895 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.666909 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:04.666958 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:04.667239 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:04.667240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:04.667397 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:04.667450 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.667452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.668381 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.668382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.668895 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.668896 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.669441 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.669441 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.669510 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.670026 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:04.670031 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:04.671396 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:04.673579 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.673579 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.673580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.673744 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.673975 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.673976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.673976 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.674029 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.674380 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:04.674381 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:04.674419 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 04:02:04.674474 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 04:02:04.674567 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 04:02:04.674816 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.666918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51192 (local address 127.31.43.253:44103)
0812 04:02:04.667087 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.667092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.667123 (+ 31us) server_negotiation.cc:407] Connection header received
0812 04:02:04.667141 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.667143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.667154 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.667240 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.667550 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.668264 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.668997 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.669343 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.669440 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.671484 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.671494 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.671495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.671503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.673485 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.673831 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.673835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.673835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.673904 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.674096 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.674097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.674097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.674303 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.674403 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.674516 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.674662 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260812 04:02:04.675472 24878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:02:04.675933 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.675790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:44988)
0812 04:02:04.675849 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.675908 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 04:02:04.676126 24825 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
I20260812 04:02:04.676580 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.676487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:44990)
0812 04:02:04.676533 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.676560 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 04:02:04.676721 24878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:34591: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.252.keytab.
Entry for principal kudu/127.31.43.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.252.keytab.
I20260812 04:02:04.685597 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.252:34591
--webserver_interface=127.31.43.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.252.keytab
--principal=kudu/127.31.43.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:43393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:33033,127.31.43.253:44103,127.31.43.252:34591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:02:04.715961 24796 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } attempt: 3
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.253@KRBTEST.COM
I20260812 04:02:04.723583 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.716473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51208 (local address 127.31.43.253:44103)
0812 04:02:04.716543 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.716545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.716557 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:02:04.716668 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.716669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.716688 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.716750 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.717131 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.717651 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.718377 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.718614 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.718748 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.720905 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.720907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.720907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.720915 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.722363 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.722766 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.722768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.722768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.722831 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.723040 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.723041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.723041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.723161 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.723242 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.723404 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.723481 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 04:02:04.723583 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.716404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:44103 (local address 127.0.0.1:51208)
0812 04:02:04.716472 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.716483 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:04.716574 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:04.716758 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:04.716759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:04.716946 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:04.717022 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.717025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.717745 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.717746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.718288 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.718289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.718728 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.718729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.718810 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.719306 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:04.719311 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:04.720813 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:04.722457 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.722458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.722458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.722641 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.722902 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.722903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.722904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.722965 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.723233 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:04.723234 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:04.723282 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 04:02:04.723365 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 04:02:04.723473 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 04:02:04.724094 24796 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:02:04.724485 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.724353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:45000)
0812 04:02:04.724410 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.724451 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 04:02:04.724871 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.724771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:45014)
0812 04:02:04.724822 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.724850 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 04:02:04.725000 24796 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:34591: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
I20260812 04:02:04.725921 24878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } attempt: 1
I20260812 04:02:04.726377 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.726268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:45022)
0812 04:02:04.726322 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.726354 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 04:02:04.726768 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.726640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:45030)
0812 04:02:04.726715 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.726741 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 04:02:04.726864 24878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:34591: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
I20260812 04:02:04.756165 24796 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } attempt: 1
I20260812 04:02:04.756692 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.756542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:45044)
0812 04:02:04.756619 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.756660 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 04:02:04.757048 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.756951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:45054)
0812 04:02:04.756995 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.757023 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 04:02:04.757155 24796 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:34591: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
W20260812 04:02:04.774210 24885 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 04:02:04.774446 24885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:02:04.774462 24885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:02:04.774483 24885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:02:04.775806 24885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:02:04.775857 24885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:02:04.775871 24885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:02:04.775884 24885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:02:04.775928 24885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:02:04.777483 24885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:43393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:33033,127.31.43.253:44103,127.31.43.252:34591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.252.keytab
--principal=kudu/127.31.43.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.252:34591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.43.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:02:04.777760 24885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:02:04.778014 24885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 04:02:04.783612 24885 server_base.cc:1034] running on GCE node
W20260812 04:02:04.783597 24891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:04.783596 24896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:04.783744 24892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:02:04.784329 24885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:02:04.784662 24885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:02:04.785818 24885 hybrid_clock.cc:648] HybridClock initialized: now 1786507324785800 us; error 35 us; skew 500 ppm
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:02:04.788830 24885 init.cc:377] Logged in from keytab as kudu/127.31.43.252@KRBTEST.COM (short username kudu)
I20260812 04:02:04.789426 24885 webserver.cc:460] Webserver started at http://127.31.43.252:35579/ using document root <none> and password file <none>
I20260812 04:02:04.789680 24885 fs_manager.cc:359] Metadata directory not provided
I20260812 04:02:04.789731 24885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:02:04.789901 24885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:02:04.791203 24885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/data/instance:
uuid: "ce71ff637ce24966969256d63545c21f"
format_stamp: "Formatted at 2026-08-12 04:02:04 on dist-test-slave-8tdl"
server_key: "9bca8f8716f65ef1ea168bbe75e12548"
server_key_iv: "c0058b03ca153bc8e62dc18f45f53778"
server_key_version: "encryptionkey@0"
I20260812 04:02:04.791631 24885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/wal/instance:
uuid: "ce71ff637ce24966969256d63545c21f"
format_stamp: "Formatted at 2026-08-12 04:02:04 on dist-test-slave-8tdl"
server_key: "9bca8f8716f65ef1ea168bbe75e12548"
server_key_iv: "c0058b03ca153bc8e62dc18f45f53778"
server_key_version: "encryptionkey@0"
I20260812 04:02:04.793222 24885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 04:02:04.794085 24904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:02:04.794297 24885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:02:04.794412 24885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/wal
uuid: "ce71ff637ce24966969256d63545c21f"
format_stamp: "Formatted at 2026-08-12 04:02:04 on dist-test-slave-8tdl"
server_key: "9bca8f8716f65ef1ea168bbe75e12548"
server_key_iv: "c0058b03ca153bc8e62dc18f45f53778"
server_key_version: "encryptionkey@0"
I20260812 04:02:04.794493 24885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:02:04.795006 24878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } attempt: 2
I20260812 04:02:04.795456 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.795315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:45062)
0812 04:02:04.795382 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.795423 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 04:02:04.795871 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.795769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:45076)
0812 04:02:04.795829 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.795854 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 04:02:04.795977 24878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:34591: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34591: connect: Connection refused (error 111)
I20260812 04:02:04.803643 24885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:02:04.806283 24885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:02:04.806471 24885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:02:04.811064 24885 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.252:34591
I20260812 04:02:04.811127 24962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.252:34591 every 8 connection(s)
I20260812 04:02:04.811489 24885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
I20260812 04:02:04.811838 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 24885
I20260812 04:02:04.811950 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786507041847525-31919-0/minicluster-data/master-2/wal/instance
I20260812 04:02:04.812196 31919 external_mini_cluster.cc:1428] Setting key b1e0a5ad3cdc74dbc03ca1945fcb0f62
I20260812 04:02:04.812928 24963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:02:04.814901 24963 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.252@KRBTEST.COM
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.252@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:02:04.823927 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.815637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35802 (local address 127.31.43.254:33033)
0812 04:02:04.815700 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.815702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.815883 (+ 181us) server_negotiation.cc:407] Connection header received
0812 04:02:04.816035 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.816037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.816050 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.816110 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.816657 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.817177 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.818169 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.818363 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.818438 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.820889 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.820891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.820891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.820902 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.822598 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.823067 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.823068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.823068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.823124 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.823317 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.823318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.823318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.823455 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.823519 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.823750 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.823829 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 04:02:04.824085 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.815561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:33033 (local address 127.0.0.1:35802)
0812 04:02:04.815762 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.815806 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:04.815887 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:04.816157 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:04.816160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:04.816496 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:04.816577 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.816579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.817278 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.817279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.818063 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.818066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.818468 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.818496 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.818587 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.819131 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:04.819141 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:04.820797 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:04.822727 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.822734 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.822736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.822996 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.823195 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.823196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.823196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.823253 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.823523 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:04.823530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:04.823618 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 04:02:04.823706 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 04:02:04.823878 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":33,"thread_start_us":110,"threads_started":1}
I20260812 04:02:04.824100 24966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.812756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.31.43.252:34591)
0812 04:02:04.812962 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.813041 (+ 79us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.813083 (+ 42us) server_negotiation.cc:407] Connection header received
0812 04:02:04.813549 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.813556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.813603 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.813755 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.814418 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.815549 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.816783 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.817196 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.817316 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.820258 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.820266 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.820267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.820274 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.822328 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.822694 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.822698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.822699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.822802 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.823020 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.823020 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.823020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.823383 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.823555 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.823639 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.823798 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260812 04:02:04.824657 24963 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.252@KRBTEST.COM for kudu/127.31.43.253@KRBTEST.COM
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:02:04.832325 24796 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } attempt: 2
I20260812 04:02:04.832458 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.825026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51216 (local address 127.31.43.253:44103)
0812 04:02:04.825078 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.825080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.825211 (+ 131us) server_negotiation.cc:407] Connection header received
0812 04:02:04.825294 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.825295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.825307 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.825360 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.825638 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.826241 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.827095 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.827379 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.827434 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.829476 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.829477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.829478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.829485 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.831100 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.831618 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.831619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.831619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.831672 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.831906 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.831906 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.831907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.832019 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.832071 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.832265 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.832368 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 04:02:04.832512 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.824929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:44103 (local address 127.0.0.1:51216)
0812 04:02:04.825169 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.825180 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:04.825236 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:04.825361 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:04.825362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:04.825504 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:04.825558 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.825559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.826319 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.826320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.826961 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.826963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.827452 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.827453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.827527 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.827912 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:04.827916 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:04.829387 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:04.831216 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.831218 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.831218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.831528 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.831751 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.831752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.831752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.831830 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.832093 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:04.832094 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:04.832153 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 04:02:04.832249 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 04:02:04.832370 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 04:02:04.832938 24963 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:02:04.833796 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.825124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35808 (local address 127.31.43.254:33033)
0812 04:02:04.825179 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.825181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.825250 (+ 69us) server_negotiation.cc:407] Connection header received
0812 04:02:04.825298 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.825299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.825309 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.825360 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.825916 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.826424 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.827221 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.827398 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.827486 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.830201 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.830202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.830202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.830210 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.832291 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.832750 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.832751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.832751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.832823 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.833084 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.833085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.833085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.833277 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.833475 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.833526 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.833634 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.252@KRBTEST.COM
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.252@KRBTEST.COM for kudu/127.31.43.252@KRBTEST.COM
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.253@KRBTEST.COM
I20260812 04:02:04.841958 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.832601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:45086)
0812 04:02:04.832652 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.832667 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:04.832711 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:04.833446 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:04.833447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:04.833692 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:04.833750 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.833752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.834923 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.834924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.835860 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.835863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.836359 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.836360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.836427 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.836818 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:04.836824 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:04.838458 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:04.840672 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.840673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.840674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.840871 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.841150 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.841151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.841152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.841225 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.841614 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:04.841615 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:04.841668 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 04:02:04.841738 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 04:02:04.841849 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 04:02:04.841958 24966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.833260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45086 (local address 127.31.43.252:34591)
0812 04:02:04.833296 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.833298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.833305 (+ 7us) server_negotiation.cc:407] Connection header received
0812 04:02:04.833328 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.833329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.833340 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.833605 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.833864 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.834820 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.835951 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.836272 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.836342 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.838560 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.838561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.838562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.838570 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.840595 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.840999 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.841000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.841000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.841067 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.841307 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.841308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.841309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.841522 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.841647 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.841775 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.841850 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 04:02:04.842651 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.833160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:45096)
0812 04:02:04.833225 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.833237 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:04.833644 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:04.833834 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:04.833835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:04.834004 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:04.834044 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.834047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.835824 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.835825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.836497 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.836500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.837044 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.837045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.837098 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.837521 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:04.837527 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:04.839279 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:04.841422 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.841423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.841424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.841704 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.841918 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.841918 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.841919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.842006 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.842302 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:04.842303 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:04.842347 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 04:02:04.842445 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 04:02:04.842545 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 04:02:04.843032 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.833261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45096 (local address 127.31.43.252:34591)
0812 04:02:04.833492 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.833494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.833502 (+ 8us) server_negotiation.cc:407] Connection header received
0812 04:02:04.833725 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.833725 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.833735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.833796 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.834142 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.835722 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.836595 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.836947 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.837023 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.839393 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.839394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.839394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.839401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.841341 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.841795 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.841796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.841796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.841848 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.842079 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.842080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.842080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.842220 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.842295 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.842841 (+ 546us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.842926 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":205,"thread_start_us":137,"threads_started":1}
I20260812 04:02:04.844113 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.834500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.31.43.253:44103)
0812 04:02:04.834546 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.834548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.834556 (+ 8us) server_negotiation.cc:407] Connection header received
0812 04:02:04.834575 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.834576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.834593 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.834661 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.835226 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.835678 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.836856 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.837132 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.837177 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.840311 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.840313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.840314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.840324 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.842362 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.843187 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.843188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.843189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.843259 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.843521 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.843522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.843523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.843657 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.843838 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.843900 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.843985 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 04:02:04.844635 24963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f: Bootstrap starting.
I20260812 04:02:04.844636 24796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68: Bootstrap starting.
I20260812 04:02:04.845445 24796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68: Neither blocks nor log segments found. Creating new log.
I20260812 04:02:04.845444 24963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f: Neither blocks nor log segments found. Creating new log.
I20260812 04:02:04.845817 24963 log.cc:824] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f: Log is configured to *not* fsync() on all Append() calls
I20260812 04:02:04.845813 24796 log.cc:824] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68: Log is configured to *not* fsync() on all Append() calls
I20260812 04:02:04.846738 24963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f: No bootstrap required, opened a new log
I20260812 04:02:04.847117 24796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68: No bootstrap required, opened a new log
I20260812 04:02:04.848269 24963 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } }
I20260812 04:02:04.848431 24963 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce71ff637ce24966969256d63545c21f, State: Initialized, Role: FOLLOWER
I20260812 04:02:04.848532 24963 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } }
I20260812 04:02:04.848735 24796 raft_consensus.cc:348] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } }
I20260812 04:02:04.848834 24972 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } } }
I20260812 04:02:04.848879 24796 raft_consensus.cc:729] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d31720c9355f4396837017449d0e4c68, State: Initialized, Role: FOLLOWER
I20260812 04:02:04.848915 24963 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:02:04.848923 24972 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:02:04.848958 24796 consensus_queue.cc:260] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } }
I20260812 04:02:04.849260 24974 sys_catalog.cc:455] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } } }
I20260812 04:02:04.849332 24796 sys_catalog.cc:564] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:02:04.849359 24974 sys_catalog.cc:458] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 04:02:04.851202 24993 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:02:04.851310 24993 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:02:04.851775 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 04:02:04.852252 25001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:02:04.852312 25001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:02:04.852798 24768 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d31720c9355f4396837017449d0e4c68"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35808
I20260812 04:02:04.852906 24768 raft_consensus.cc:482] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 04:02:04.852959 24768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:02:04.853945 24768 raft_consensus.cc:504] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } }
I20260812 04:02:04.854164 24768 leader_election.cc:288] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [CANDIDATE]: Term 1 election: Requested vote from peers 6ebdb9bc2519475fbbd4a61360377229 (127.31.43.253:44103), ce71ff637ce24966969256d63545c21f (127.31.43.252:34591)
I20260812 04:02:04.854439 24935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d31720c9355f4396837017449d0e4c68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ce71ff637ce24966969256d63545c21f"
I20260812 04:02:04.854557 24935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:02:04.852042 24722 master_runner.cc:186] Error getting master registration for 127.31.43.253:44103: OK, instance_id { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" instance_seqno: 1786507324649574 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 04:02:04.855041 24722 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 04:02:04.855518 24935 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d31720c9355f4396837017449d0e4c68 in term 1.
I20260812 04:02:04.852908 24885 master_runner.cc:186] Error getting master registration for 127.31.43.253:44103: OK, instance_id { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" instance_seqno: 1786507324649574 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 04:02:04.855654 24885 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 04:02:04.855830 24743 leader_election.cc:302] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce71ff637ce24966969256d63545c21f, d31720c9355f4396837017449d0e4c68; no voters:
I20260812 04:02:04.855967 24974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:02:04.856189 24974 raft_consensus.cc:686] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [term 1 LEADER]: Becoming Leader. State: Replica: d31720c9355f4396837017449d0e4c68, State: Running, Role: LEADER
I20260812 04:02:04.856295 24974 consensus_queue.cc:237] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } }
I20260812 04:02:04.856791 25003 sys_catalog.cc:455] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d31720c9355f4396837017449d0e4c68. Latest consensus state: current_term: 1 leader_uuid: "d31720c9355f4396837017449d0e4c68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } } }
I20260812 04:02:04.856969 25003 sys_catalog.cc:458] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [sys.catalog]: This master's current role is: LEADER
I20260812 04:02:04.862171 25011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:02:04.862470 25011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 04:02:04.864353 24935 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [term 1 FOLLOWER]: Refusing update from remote peer d31720c9355f4396837017449d0e4c68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:02:04.864744 25003 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:02:04.866251 24972 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d31720c9355f4396837017449d0e4c68. Latest consensus state: current_term: 1 leader_uuid: "d31720c9355f4396837017449d0e4c68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } } }
I20260812 04:02:04.866365 24972 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:02:04.867106 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.856875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45102 (local address 127.31.43.252:34591)
0812 04:02:04.856920 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.856922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.857243 (+ 321us) server_negotiation.cc:407] Connection header received
0812 04:02:04.857349 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.857350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.857362 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.857394 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.858208 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.859006 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.860167 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.860495 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.860538 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.862949 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.862951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.862951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.862959 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.865031 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.865464 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.865465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.865466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.865538 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.866012 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.866013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.866014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.866196 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.866282 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.866890 (+ 608us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.866990 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 04:02:04.868227 25003 sys_catalog.cc:455] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d31720c9355f4396837017449d0e4c68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } } }
I20260812 04:02:04.868319 25003 sys_catalog.cc:458] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [sys.catalog]: This master's current role is: LEADER
I20260812 04:02:04.868801 24972 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d31720c9355f4396837017449d0e4c68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } } }
I20260812 04:02:04.868883 24972 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:02:04.869136 25014 mvcc.cc:205] Tried to move back new op lower bound from 7317534002641387520 to 7317534002612244480. Current Snapshot: MvccSnapshot[applied={T|T < 7317534002641387520}]
I20260812 04:02:04.869676 25015 mvcc.cc:205] Tried to move back new op lower bound from 7317534002641387520 to 7317534002612244480. Current Snapshot: MvccSnapshot[applied={T|T < 7317534002641387520}]
I20260812 04:02:04.870007 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.857153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35820 (local address 127.31.43.254:33033)
0812 04:02:04.857211 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.857212 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.857433 (+ 221us) server_negotiation.cc:407] Connection header received
0812 04:02:04.857610 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.857612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.857623 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.857658 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.859012 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.859735 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.861186 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.861479 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.861524 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.864078 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.864080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.864080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.864088 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.866469 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.868800 (+ 2331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.868801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.868802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.868865 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.869204 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.869205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.869205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.869348 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.869447 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.869777 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.869877 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 04:02:04.870739 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.857508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51226 (local address 127.31.43.253:44103)
0812 04:02:04.857551 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.857553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.858339 (+ 786us) server_negotiation.cc:407] Connection header received
0812 04:02:04.858451 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.858452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.858465 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.858510 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.859251 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.859935 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.860898 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.861219 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.861258 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.866525 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.866527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.866527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.866536 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.868360 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.869027 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.869028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.869029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.869083 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.869380 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.869381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.869382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.869542 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.870133 (+ 591us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.870498 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.870599 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 04:02:04.871201 25011 catalog_manager.cc:1347] Generated new cluster ID: da5616b14dd348118bb521063ed68afc
I20260812 04:02:04.871258 25011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:02:04.887101 24878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } attempt: 3
Aug 12 04:02:04 dist-test-slave-8tdl krb5kdc[24709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507324, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.253@KRBTEST.COM for kudu/127.31.43.252@KRBTEST.COM
I20260812 04:02:04.896843 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.887449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34591 (local address 127.0.0.1:45116)
0812 04:02:04.887549 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:04.887560 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:04.887597 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:04.887831 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:04.887832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:04.888041 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:04.888090 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.888092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.889700 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.889701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.890501 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:04.890503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.891086 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.891087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:04.891142 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.891638 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:04.891644 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:04.893087 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:04.894939 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.894940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.894940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.895160 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.896214 (+ 1054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:04.896215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:04.896216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:04.896275 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:04.896528 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:04.896529 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:04.896575 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 04:02:04.896632 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 04:02:04.896740 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 04:02:04.897184 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:04.887662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.31.43.252:34591)
0812 04:02:04.887699 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.887700 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.887707 (+ 7us) server_negotiation.cc:407] Connection header received
0812 04:02:04.887732 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.887733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.887743 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.887786 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.888164 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.888890 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.890641 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:04.890971 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:04.891028 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:04.893178 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:04.893179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:04.893180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:04.893187 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:04.894835 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.896075 (+ 1240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.896076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.896077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.896146 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:04.896330 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:04.896331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:04.896331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:04.896446 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:04.896506 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:04.897021 (+ 515us) server_negotiation.cc:299] Negotiation successful
0812 04:02:04.897098 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 04:02:04.898890 24878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229: Bootstrap starting.
I20260812 04:02:04.899730 24878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229: Neither blocks nor log segments found. Creating new log.
I20260812 04:02:04.900090 24878 log.cc:824] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229: Log is configured to *not* fsync() on all Append() calls
I20260812 04:02:04.901059 24878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229: No bootstrap required, opened a new log
I20260812 04:02:04.903350 24878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } }
I20260812 04:02:04.903530 24878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebdb9bc2519475fbbd4a61360377229, State: Initialized, Role: FOLLOWER
I20260812 04:02:04.903625 24878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } }
I20260812 04:02:04.904120 24850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:02:04.904016 24878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:02:04.905200 24850 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [term 1 FOLLOWER]: Refusing update from remote peer d31720c9355f4396837017449d0e4c68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:02:04.905396 24852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d31720c9355f4396837017449d0e4c68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6ebdb9bc2519475fbbd4a61360377229"
I20260812 04:02:04.905530 25003 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:02:04.906345 24852 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d31720c9355f4396837017449d0e4c68 in term 1.
I20260812 04:02:04.907019 25011 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:02:04.904002 25020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } } }
I20260812 04:02:04.907873 25020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:02:04.907836 25011 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:02:04.907997 25020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d31720c9355f4396837017449d0e4c68. Latest consensus state: current_term: 1 leader_uuid: "d31720c9355f4396837017449d0e4c68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } } }
I20260812 04:02:04.908075 25020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:02:04.909294 25020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d31720c9355f4396837017449d0e4c68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } } }
I20260812 04:02:04.909368 25020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 04:02:04.909969 25032 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:02:04.910022 25032 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:02:04.911214 25035 mvcc.cc:205] Tried to move back new op lower bound from 7317534002788589568 to 7317534002612244480. Current Snapshot: MvccSnapshot[applied={T|T < 7317534002641387520}]
I20260812 04:02:04.913950 25003 sys_catalog.cc:455] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d31720c9355f4396837017449d0e4c68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d31720c9355f4396837017449d0e4c68" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 33033 } } peers { permanent_uuid: "6ebdb9bc2519475fbbd4a61360377229" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 44103 } } peers { permanent_uuid: "ce71ff637ce24966969256d63545c21f" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34591 } } }
I20260812 04:02:04.914067 25003 sys_catalog.cc:458] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68 [sys.catalog]: This master's current role is: LEADER
I20260812 04:02:04.915655 25011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d31720c9355f4396837017449d0e4c68: Generated new TSK 0
I20260812 04:02:04.915899 25011 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 04:02:04.946002 24968 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:04.944472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45120 (local address 127.31.43.252:34591)
0812 04:02:04.944527 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.944530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.945242 (+ 712us) server_negotiation.cc:407] Connection header received
0812 04:02:04.945361 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.945363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.945374 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.945405 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.945911 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45120: BlockingRecv error: recv got EOF from 127.0.0.1:45120 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 04:02:04.947036 24800 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:04.944900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35826 (local address 127.31.43.254:33033)
0812 04:02:04.944944 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.944945 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.946435 (+ 1490us) server_negotiation.cc:407] Connection header received
0812 04:02:04.946537 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.946538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.946550 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.946581 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.946955 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35826: BlockingRecv error: recv got EOF from 127.0.0.1:35826 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 04:02:04.948246 24882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:04.947567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51228 (local address 127.31.43.253:44103)
0812 04:02:04.947609 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.947611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.947720 (+ 109us) server_negotiation.cc:407] Connection header received
0812 04:02:04.947816 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.947817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.947828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.947857 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.948185 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51228: BlockingRecv error: recv got EOF from 127.0.0.1:51228 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 04:02:04.951150 24882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:04.950279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.31.43.253:44103)
0812 04:02:04.950316 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.950318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.950544 (+ 226us) server_negotiation.cc:407] Connection header received
0812 04:02:04.950658 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.950659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.950711 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.950742 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.951099 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51232: BlockingRecv error: recv got EOF from 127.0.0.1:51232 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 04:02:04.954384 24882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:04.953613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51242 (local address 127.31.43.253:44103)
0812 04:02:04.953648 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.953650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.953865 (+ 215us) server_negotiation.cc:407] Connection header received
0812 04:02:04.953962 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.953963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.953973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.954013 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.954329 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51242: BlockingRecv error: recv got EOF from 127.0.0.1:51242 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 04:02:04.958568 24882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:04.957753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51246 (local address 127.31.43.253:44103)
0812 04:02:04.957792 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.957794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.958026 (+ 232us) server_negotiation.cc:407] Connection header received
0812 04:02:04.958130 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.958131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.958141 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.958170 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.958515 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51246: BlockingRecv error: recv got EOF from 127.0.0.1:51246 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 04:02:04.964979 24882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:04.964137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51258 (local address 127.31.43.253:44103)
0812 04:02:04.964178 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.964180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.964412 (+ 232us) server_negotiation.cc:407] Connection header received
0812 04:02:04.964517 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.964518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.964530 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.964561 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.964925 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51258: BlockingRecv error: recv got EOF from 127.0.0.1:51258 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 04:02:04.975639 24882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:04.974510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51260 (local address 127.31.43.253:44103)
0812 04:02:04.974555 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.974557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.974955 (+ 398us) server_negotiation.cc:407] Connection header received
0812 04:02:04.975078 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.975079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.975091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.975123 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.975547 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51260: BlockingRecv error: recv got EOF from 127.0.0.1:51260 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 04:02:04.994338 24882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:04.993324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51262 (local address 127.31.43.253:44103)
0812 04:02:04.993371 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:04.993373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:04.993658 (+ 285us) server_negotiation.cc:407] Connection header received
0812 04:02:04.993771 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:04.993772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:04.993784 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:04.993817 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:04.994276 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51262: BlockingRecv error: recv got EOF from 127.0.0.1:51262 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 04:02:05.029258 24882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:05.028162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51264 (local address 127.31.43.253:44103)
0812 04:02:05.028216 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:05.028218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:05.028583 (+ 365us) server_negotiation.cc:407] Connection header received
0812 04:02:05.028705 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:05.028707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:05.028719 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:05.028754 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:05.029192 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51264: BlockingRecv error: recv got EOF from 127.0.0.1:51264 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 04:02:05.095793 24882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:05.095123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51266 (local address 127.31.43.253:44103)
0812 04:02:05.095185 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:05.095187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:05.095269 (+ 82us) server_negotiation.cc:407] Connection header received
0812 04:02:05.095323 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:05.095324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:05.095336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:05.095390 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:05.095742 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51266: BlockingRecv error: recv got EOF from 127.0.0.1:51266 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260812 04:02:05.226013 24882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:05.225281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51270 (local address 127.31.43.253:44103)
0812 04:02:05.225370 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:05.225372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:05.225435 (+ 63us) server_negotiation.cc:407] Connection header received
0812 04:02:05.225504 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:05.225505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:05.225517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:05.225562 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:05.225921 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51270: BlockingRecv error: recv got EOF from 127.0.0.1:51270 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 04:02:05.484230 24882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:05.483544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51272 (local address 127.31.43.253:44103)
0812 04:02:05.483634 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:05.483636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:05.483647 (+ 11us) server_negotiation.cc:407] Connection header received
0812 04:02:05.483679 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:05.483680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:05.483690 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:05.483760 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:05.484175 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51272: BlockingRecv error: recv got EOF from 127.0.0.1:51272 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 04:02:05.851850 24993 catalog_manager.cc:1259] Loaded cluster ID: da5616b14dd348118bb521063ed68afc
I20260812 04:02:05.851962 24993 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f: loading cluster ID for follower catalog manager: success
I20260812 04:02:05.852803 24993 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f: acquiring CA information for follower catalog manager: success
I20260812 04:02:05.853164 24993 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ce71ff637ce24966969256d63545c21f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 04:02:05.910661 25032 catalog_manager.cc:1259] Loaded cluster ID: da5616b14dd348118bb521063ed68afc
I20260812 04:02:05.910789 25032 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229: loading cluster ID for follower catalog manager: success
I20260812 04:02:05.911693 25032 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229: acquiring CA information for follower catalog manager: success
I20260812 04:02:05.912014 25032 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6ebdb9bc2519475fbbd4a61360377229: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 04:02:06.000707 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:05.997560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51276 (local address 127.31.43.253:44103)
0812 04:02:05.997755 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:05.997757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:05.997768 (+ 11us) server_negotiation.cc:407] Connection header received
0812 04:02:05.997799 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:05.997800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:05.997853 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:05.998020 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:02:05.998179 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:05.998821 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:05.999662 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:05.999954 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:06.000031 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:06.000289 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:02:06.000418 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:02:06.000504 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:06.000540 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 04:02:06.000563 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":128,"threads_started":1}
I20260812 04:02:06.005703 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:06.003018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45124 (local address 127.31.43.252:34591)
0812 04:02:06.003133 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:06.003135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:06.003147 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:02:06.003217 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:06.003217 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:06.003256 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:06.003326 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:02:06.003414 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:06.003953 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:06.004744 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:06.004992 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:06.005061 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:06.005250 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:02:06.005393 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:02:06.005474 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:06.005509 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 04:02:06.005534 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":63,"threads_started":1}
I20260812 04:02:06.010625 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:06.007811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45128 (local address 127.31.43.252:34591)
0812 04:02:06.007854 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:06.007856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:06.008020 (+ 164us) server_negotiation.cc:407] Connection header received
0812 04:02:06.008122 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:06.008123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:06.008132 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:06.008165 (+ 33us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:02:06.008395 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:06.009092 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:06.009873 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:06.010150 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:06.010218 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:06.010347 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:02:06.010393 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:02:06.010444 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:06.010519 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 04:02:06.010542 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 04:02:06.010892 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:06.007642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.31.43.253:44103)
0812 04:02:06.007691 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:06.007694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:06.007936 (+ 242us) server_negotiation.cc:407] Connection header received
0812 04:02:06.008008 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:06.008009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:06.008022 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:06.008068 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:02:06.008172 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:06.008949 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:06.009959 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:06.010276 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:06.010359 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:06.010538 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:02:06.010577 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:02:06.010608 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:06.010764 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 04:02:06.010790 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 04:02:06.011826 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:06.007833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35832 (local address 127.31.43.254:33033)
0812 04:02:06.007945 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:06.007947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:06.007954 (+ 7us) server_negotiation.cc:407] Connection header received
0812 04:02:06.008008 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:06.008009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:06.008073 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:06.008245 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:02:06.008343 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:06.008851 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:06.010405 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:06.010741 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:06.011314 (+ 573us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:06.011405 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:02:06.011527 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:02:06.011585 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:06.011669 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 04:02:06.011692 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":62,"threads_started":1}
I20260812 04:02:06.014098 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 24722
I20260812 04:02:06.024755 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 24804
I20260812 04:02:06.032577 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 24885
2026-08-12T04:02:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2953 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 04:02:06 dist-test-slave-8tdl krb5kdc[25131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 04:02:06 dist-test-slave-8tdl krb5kdc[25131](info): set up 2 sockets
Aug 12 04:02:06 dist-test-slave-8tdl krb5kdc[25131](info): commencing operation
krb5kdc: starting...
W20260812 04:02:07.181327 31919 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/test-user.keytab.
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T04:02:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T04:02:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.43.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
Entry for principal kudu/127.31.43.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab.
I20260812 04:02:07.322665 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:35595
--webserver_interface=127.31.43.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:35595,127.31.43.253:46791,127.31.43.252:34071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 04:02:07.410419 25147 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 04:02:07.410667 25147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:02:07.410713 25147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:02:07.410772 25147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:02:07.412036 25147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:02:07.412084 25147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:02:07.412099 25147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:02:07.412113 25147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:02:07.412125 25147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:02:07.413760 25147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:35595,127.31.43.253:46791,127.31.43.252:34071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.254.keytab
--principal=kudu/127.31.43.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.254:35595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.43.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:02:07.414062 25147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:02:07.414345 25147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 04:02:07.419884 25152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:07.420096 25156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:07.419999 25153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:02:07.420239 25147 server_base.cc:1034] running on GCE node
I20260812 04:02:07.420508 25147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:02:07.420789 25147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:02:07.421962 25147 hybrid_clock.cc:648] HybridClock initialized: now 1786507327421939 us; error 37 us; skew 500 ppm
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:02:07.424814 25147 init.cc:377] Logged in from keytab as kudu/127.31.43.254@KRBTEST.COM (short username kudu)
I20260812 04:02:07.425328 25147 webserver.cc:460] Webserver started at http://127.31.43.254:38951/ using document root <none> and password file <none>
I20260812 04:02:07.425562 25147 fs_manager.cc:359] Metadata directory not provided
I20260812 04:02:07.425608 25147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:02:07.425705 25147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:02:07.426900 25147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/data/instance:
uuid: "5b357bca90bd495b93426519842238ef"
format_stamp: "Formatted at 2026-08-12 04:02:07 on dist-test-slave-8tdl"
server_key: "2484304c746374f9a31be8cd0d86d478"
server_key_iv: "ef1df89506114370807f855db0f8562f"
server_key_version: "encryptionkey@0"
I20260812 04:02:07.427243 25147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/wal/instance:
uuid: "5b357bca90bd495b93426519842238ef"
format_stamp: "Formatted at 2026-08-12 04:02:07 on dist-test-slave-8tdl"
server_key: "2484304c746374f9a31be8cd0d86d478"
server_key_iv: "ef1df89506114370807f855db0f8562f"
server_key_version: "encryptionkey@0"
I20260812 04:02:07.428674 25147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:02:07.429627 25164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:02:07.429860 25147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 04:02:07.429955 25147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/wal
uuid: "5b357bca90bd495b93426519842238ef"
format_stamp: "Formatted at 2026-08-12 04:02:07 on dist-test-slave-8tdl"
server_key: "2484304c746374f9a31be8cd0d86d478"
server_key_iv: "ef1df89506114370807f855db0f8562f"
server_key_version: "encryptionkey@0"
I20260812 04:02:07.430029 25147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:02:07.443962 25147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:02:07.446441 25147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:02:07.446584 25147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:02:07.450944 25147 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.254:35595
I20260812 04:02:07.451036 25220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.254:35595 every 8 connection(s)
I20260812 04:02:07.451417 25147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/data/info.pb
I20260812 04:02:07.452512 25221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:02:07.454217 25221 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:02:07.459574 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 25147
I20260812 04:02:07.459721 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-0/wal/instance
I20260812 04:02:07.459972 31919 external_mini_cluster.cc:1428] Setting key 0eae1a665e495ed38931c2e727acfe52
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:02:07.465006 25226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.455193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49892 (local address 127.31.43.254:35595)
0812 04:02:07.455386 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.455395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.455478 (+ 83us) server_negotiation.cc:407] Connection header received
0812 04:02:07.456077 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.456079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.456097 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.456185 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.456802 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.457632 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.458550 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.458948 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.459038 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.461522 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.461530 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.461530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.461540 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.463259 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.463755 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.463759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.463759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.463848 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.464170 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.464171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.464171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.464406 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.464481 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.464722 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.464827 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
I20260812 04:02:07.465160 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.455133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35595 (local address 127.0.0.1:49892)
0812 04:02:07.455335 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.455380 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:07.455907 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:07.456254 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:07.456257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:07.456532 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:07.456700 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.456704 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.457721 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.457722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.458462 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.458464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.459053 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.459054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.459113 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.459687 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:07.459699 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:07.461420 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:07.463369 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.463377 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.463381 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.463658 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.463934 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.463935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.463935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.464055 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.464483 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:07.464491 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:07.464574 (+ 83us) client_negotiation.cc:769] Sending connection context
0812 04:02:07.464678 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 04:02:07.464954 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":73,"thread_start_us":95,"threads_started":1}
I20260812 04:02:07.466022 25221 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:02:07.466447 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.466302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:46791 (local address 127.0.0.1:52198)
0812 04:02:07.466360 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.466415 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 04:02:07.466611 25166 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.43.253:46791: connect: Connection refused (error 111)
I20260812 04:02:07.467147 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.467028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:46791 (local address 127.0.0.1:52212)
0812 04:02:07.467082 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.467116 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 04:02:07.467331 25221 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.253:46791: Network error: Client connection negotiation failed: client connection to 127.31.43.253:46791: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.31.43.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.253.keytab.
Entry for principal kudu/127.31.43.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.253.keytab.
I20260812 04:02:07.493561 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.253:46791
--webserver_interface=127.31.43.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.253.keytab
--principal=kudu/127.31.43.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:35595,127.31.43.253:46791,127.31.43.252:34071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:02:07.516615 25221 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } attempt: 1
I20260812 04:02:07.517315 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.517147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:46791 (local address 127.0.0.1:52226)
0812 04:02:07.517223 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.517274 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 04:02:07.517761 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.517662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:46791 (local address 127.0.0.1:52230)
0812 04:02:07.517710 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.517742 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 04:02:07.517901 25221 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.253:46791: Network error: Client connection negotiation failed: client connection to 127.31.43.253:46791: connect: Connection refused (error 111)
W20260812 04:02:07.582345 25230 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 04:02:07.582600 25230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:02:07.582620 25230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:02:07.582643 25230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:02:07.583942 25230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:02:07.583995 25230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:02:07.584010 25230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:02:07.584084 25230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:02:07.584113 25230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:02:07.585785 25230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:35595,127.31.43.253:46791,127.31.43.252:34071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.253.keytab
--principal=kudu/127.31.43.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.253:46791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.43.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:02:07.586084 25230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:02:07.586174 25221 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } attempt: 2
I20260812 04:02:07.586373 25230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 04:02:07.586731 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.586537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:46791 (local address 127.0.0.1:52232)
0812 04:02:07.586612 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.586662 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 04:02:07.587131 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.587019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:46791 (local address 127.0.0.1:52246)
0812 04:02:07.587073 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.587107 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.253:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 04:02:07.587253 25221 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.253:46791: Network error: Client connection negotiation failed: client connection to 127.31.43.253:46791: connect: Connection refused (error 111)
W20260812 04:02:07.592350 25235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:07.592339 25239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:02:07.592505 25230 server_base.cc:1034] running on GCE node
W20260812 04:02:07.592345 25237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:02:07.592934 25230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 04:02:07.593291 25230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:02:07.594511 25230 hybrid_clock.cc:648] HybridClock initialized: now 1786507327594481 us; error 52 us; skew 500 ppm
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:02:07.597672 25230 init.cc:377] Logged in from keytab as kudu/127.31.43.253@KRBTEST.COM (short username kudu)
I20260812 04:02:07.598289 25230 webserver.cc:460] Webserver started at http://127.31.43.253:46511/ using document root <none> and password file <none>
I20260812 04:02:07.598578 25230 fs_manager.cc:359] Metadata directory not provided
I20260812 04:02:07.598661 25230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:02:07.598867 25230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:02:07.600203 25230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/data/instance:
uuid: "885cd073447a4577bd2b21c513e86192"
format_stamp: "Formatted at 2026-08-12 04:02:07 on dist-test-slave-8tdl"
server_key: "c58109ca651da8faa38ba5cdc22aa243"
server_key_iv: "863672eca3b933dca6a1d7c4e5b445c4"
server_key_version: "encryptionkey@0"
I20260812 04:02:07.600627 25230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/wal/instance:
uuid: "885cd073447a4577bd2b21c513e86192"
format_stamp: "Formatted at 2026-08-12 04:02:07 on dist-test-slave-8tdl"
server_key: "c58109ca651da8faa38ba5cdc22aa243"
server_key_iv: "863672eca3b933dca6a1d7c4e5b445c4"
server_key_version: "encryptionkey@0"
I20260812 04:02:07.602370 25230 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 04:02:07.603329 25246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:02:07.603516 25230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 04:02:07.603644 25230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/wal
uuid: "885cd073447a4577bd2b21c513e86192"
format_stamp: "Formatted at 2026-08-12 04:02:07 on dist-test-slave-8tdl"
server_key: "c58109ca651da8faa38ba5cdc22aa243"
server_key_iv: "863672eca3b933dca6a1d7c4e5b445c4"
server_key_version: "encryptionkey@0"
I20260812 04:02:07.603729 25230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:02:07.616611 25230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:02:07.618942 25230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:02:07.619119 25230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:02:07.623736 25230 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.253:46791
I20260812 04:02:07.623785 25307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.253:46791 every 8 connection(s)
I20260812 04:02:07.624179 25230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/data/info.pb
I20260812 04:02:07.625342 25308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:02:07.627097 25308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:02:07.630100 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 25230
I20260812 04:02:07.630252 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-1/wal/instance
I20260812 04:02:07.630494 31919 external_mini_cluster.cc:1428] Setting key efab23e04f3782d089a18fe7e8008869
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.253@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:02:07.638463 25226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.627840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49908 (local address 127.31.43.254:35595)
0812 04:02:07.627992 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.627994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.628169 (+ 175us) server_negotiation.cc:407] Connection header received
0812 04:02:07.628624 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.628625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.628635 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.628701 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.629415 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.629995 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.631384 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.631725 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.631804 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.634935 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.634938 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.634938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.634948 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.636797 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.637330 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.637332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.637332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.637406 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.637709 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.637710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.637711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.637888 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.637971 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.638240 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.638346 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 04:02:07.638660 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.627811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35595 (local address 127.0.0.1:49908)
0812 04:02:07.628023 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.628067 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:07.628533 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:07.628782 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:07.628785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:07.629122 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:07.629329 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.629334 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.630138 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.630140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.631288 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.631291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.631836 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.631837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.632017 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.632646 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:07.632658 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:07.634795 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:07.636930 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.636939 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.636941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.637235 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.637483 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.637484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.637484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.637628 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.637961 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:07.637966 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:07.638103 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 04:02:07.638190 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 04:02:07.638338 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":123,"threads_started":1}
I20260812 04:02:07.639345 25308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.43.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.43.252@KRBTEST.COM" created.
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.253@KRBTEST.COM for kudu/127.31.43.253@KRBTEST.COM
I20260812 04:02:07.648072 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.639675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:46791 (local address 127.0.0.1:52252)
0812 04:02:07.639727 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.639738 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:07.639782 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:07.640084 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:07.640085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:07.640257 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:07.640335 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.640337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.641333 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.641334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.641903 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.641905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.642388 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.642389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.642448 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.642914 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:07.642919 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:07.644844 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:07.646792 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.646794 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.646794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.647042 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.647361 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.647362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.647363 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.647447 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.647739 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:07.647740 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:07.647781 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 04:02:07.647839 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 04:02:07.647973 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 04:02:07.648243 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.639776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52252 (local address 127.31.43.253:46791)
0812 04:02:07.639943 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.639947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.639965 (+ 18us) server_negotiation.cc:407] Connection header received
0812 04:02:07.639981 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.639982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.639994 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.640046 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.640422 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.641250 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.641984 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.642314 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.642375 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.644953 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.644963 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.644963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.644969 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.646657 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.647212 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.647216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.647216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.647277 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.647520 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.647521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.647521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.647675 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.647748 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.647914 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.648066 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":140,"thread_start_us":112,"threads_started":1}
I20260812 04:02:07.648964 25308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:02:07.649355 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.649232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46696)
0812 04:02:07.649283 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.649335 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 04:02:07.649539 25248 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
I20260812 04:02:07.650036 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.649932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46712)
0812 04:02:07.649986 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.650019 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 04:02:07.650170 25308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:34071: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.43.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.252.keytab.
Entry for principal kudu/127.31.43.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.252.keytab.
I20260812 04:02:07.665865 31919 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaEzvCg/build/release/bin/kudu
/tmp/dist-test-taskaEzvCg/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.252:34071
--webserver_interface=127.31.43.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.252.keytab
--principal=kudu/127.31.43.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.43.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.43.254:35595,127.31.43.253:46791,127.31.43.252:34071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 04:02:07.678421 25221 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } attempt: 3
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.253@KRBTEST.COM
I20260812 04:02:07.686600 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.678978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52262 (local address 127.31.43.253:46791)
0812 04:02:07.679047 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.679049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.679063 (+ 14us) server_negotiation.cc:407] Connection header received
0812 04:02:07.679147 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.679148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.679163 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.679244 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.679587 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.680390 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.681164 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.681516 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.681593 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.683750 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.683751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.683752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.683759 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.685447 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.685860 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.685861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.685861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.685922 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.686127 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.686127 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.686128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.686241 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.686294 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.686453 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.686529 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 04:02:07.686645 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.678868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:46791 (local address 127.0.0.1:52262)
0812 04:02:07.678942 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.678956 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:07.679069 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:07.679257 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:07.679258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:07.679418 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:07.679521 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.679523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.680483 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.680484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.681068 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.681070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.681589 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.681590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.681654 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.682200 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:07.682205 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:07.683629 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:07.685538 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.685539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.685539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.685767 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.685995 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.685996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.685997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.686064 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.686304 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:07.686305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:07.686356 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 04:02:07.686425 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 04:02:07.686524 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 04:02:07.687269 25221 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:02:07.687628 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.687524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46722)
0812 04:02:07.687575 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.687606 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 04:02:07.687904 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.687856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46734)
0812 04:02:07.687876 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.687894 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260812 04:02:07.688059 25221 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:34071: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
I20260812 04:02:07.699461 25308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } attempt: 1
I20260812 04:02:07.699990 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.699844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46736)
0812 04:02:07.699908 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.699958 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 04:02:07.700348 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.700250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46744)
0812 04:02:07.700298 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.700322 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 04:02:07.700434 25308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:34071: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
I20260812 04:02:07.719213 25221 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } attempt: 1
I20260812 04:02:07.719750 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.719598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46754)
0812 04:02:07.719671 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.719717 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 04:02:07.720136 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.720025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46764)
0812 04:02:07.720076 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.720104 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 04:02:07.720232 25221 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:34071: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
W20260812 04:02:07.760344 25315 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 04:02:07.760635 25315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 04:02:07.760670 25315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 04:02:07.760715 25315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 04:02:07.761960 25315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 04:02:07.762038 25315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 04:02:07.762063 25315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 04:02:07.762099 25315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 04:02:07.762130 25315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 04:02:07.763751 25315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.43.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.43.254:35595,127.31.43.253:46791,127.31.43.252:34071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/krb5kdc/kudu_127.31.43.252.keytab
--principal=kudu/127.31.43.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.43.252:34071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.43.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 04:02:07.764052 25315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 04:02:07.764302 25315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 04:02:07.768602 25308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } attempt: 2
I20260812 04:02:07.769296 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.769075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46770)
0812 04:02:07.769212 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.769255 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260812 04:02:07.769451 25324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:07.769487 25322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 04:02:07.769476 25321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 04:02:07.769637 25315 server_base.cc:1034] running on GCE node
I20260812 04:02:07.769877 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.769773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46782)
0812 04:02:07.769816 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.769854 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 04:02:07.769940 25315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 04:02:07.769980 25308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:34071: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
I20260812 04:02:07.770288 25315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 04:02:07.771433 25315 hybrid_clock.cc:648] HybridClock initialized: now 1786507327771402 us; error 54 us; skew 500 ppm
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 04:02:07.774191 25315 init.cc:377] Logged in from keytab as kudu/127.31.43.252@KRBTEST.COM (short username kudu)
I20260812 04:02:07.774793 25315 webserver.cc:460] Webserver started at http://127.31.43.252:33563/ using document root <none> and password file <none>
I20260812 04:02:07.775046 25315 fs_manager.cc:359] Metadata directory not provided
I20260812 04:02:07.775100 25315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 04:02:07.775265 25315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 04:02:07.776396 25315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/data/instance:
uuid: "99032a6e18374f6b88c29dbb15cb838e"
format_stamp: "Formatted at 2026-08-12 04:02:07 on dist-test-slave-8tdl"
server_key: "8231b25cc99ed6c22c9347de4ea92b6e"
server_key_iv: "f894e42afc965c3ddb0605e5d1c7bade"
server_key_version: "encryptionkey@0"
I20260812 04:02:07.776782 25315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/wal/instance:
uuid: "99032a6e18374f6b88c29dbb15cb838e"
format_stamp: "Formatted at 2026-08-12 04:02:07 on dist-test-slave-8tdl"
server_key: "8231b25cc99ed6c22c9347de4ea92b6e"
server_key_iv: "f894e42afc965c3ddb0605e5d1c7bade"
server_key_version: "encryptionkey@0"
I20260812 04:02:07.778239 25315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 04:02:07.779102 25331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 04:02:07.779309 25315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 04:02:07.779404 25315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/data,/tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/wal
uuid: "99032a6e18374f6b88c29dbb15cb838e"
format_stamp: "Formatted at 2026-08-12 04:02:07 on dist-test-slave-8tdl"
server_key: "8231b25cc99ed6c22c9347de4ea92b6e"
server_key_iv: "f894e42afc965c3ddb0605e5d1c7bade"
server_key_version: "encryptionkey@0"
I20260812 04:02:07.779479 25315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 04:02:07.790231 25315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 04:02:07.793125 25315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 04:02:07.793272 25315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 04:02:07.795393 25221 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } attempt: 2
I20260812 04:02:07.795893 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.795768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46786)
0812 04:02:07.795815 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.795850 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 04:02:07.796320 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.796149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46802)
0812 04:02:07.796227 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.796276 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 04:02:07.796444 25221 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.43.252:34071: Network error: Client connection negotiation failed: client connection to 127.31.43.252:34071: connect: Connection refused (error 111)
I20260812 04:02:07.798285 25315 rpc_server.cc:307] RPC server started. Bound to: 127.31.43.252:34071
I20260812 04:02:07.798314 25391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.43.252:34071 every 8 connection(s)
I20260812 04:02:07.798840 25315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/data/info.pb
I20260812 04:02:07.800025 25392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 04:02:07.801808 25392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:02:07.803053 31919 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaEzvCg/build/release/bin/kudu as pid 25315
I20260812 04:02:07.803187 31919 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaEzvCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786507041847525-31919-0/minicluster-data/master-2/wal/instance
I20260812 04:02:07.803380 31919 external_mini_cluster.cc:1428] Setting key a81b9876e3b4fce806b96df464830144
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.252@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.252@KRBTEST.COM
I20260812 04:02:07.812912 25226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.802598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49920 (local address 127.31.43.254:35595)
0812 04:02:07.802703 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.802705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.802974 (+ 269us) server_negotiation.cc:407] Connection header received
0812 04:02:07.803589 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.803591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.803603 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.803666 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.804565 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.805253 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.806640 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.806895 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.806972 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.809506 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.809507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.809507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.809516 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.811295 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.811788 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.811789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.811789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.811842 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.812121 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.812122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.812122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.812270 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.812348 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.812717 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.812790 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 04:02:07.813120 25394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.802574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.254:35595 (local address 127.0.0.1:49920)
0812 04:02:07.802841 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.802881 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:07.803440 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:07.803760 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:07.803769 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:07.804187 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:07.804458 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.804463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.805433 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.805434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.806526 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.806528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.807015 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.807016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.807189 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.807810 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:07.807826 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:07.809405 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:07.811380 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.811385 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.811388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.811704 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.811904 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.811905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.811905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.812047 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.812353 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:07.812359 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:07.812558 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 04:02:07.812686 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 04:02:07.812859 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":3,"thread_start_us":153,"threads_started":1}
I20260812 04:02:07.813643 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.803791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46810 (local address 127.31.43.252:34071)
0812 04:02:07.803998 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.804003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.804029 (+ 26us) server_negotiation.cc:407] Connection header received
0812 04:02:07.804105 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.804108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.804133 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.804219 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.805022 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.806309 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.807155 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.807557 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.807612 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.810068 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.810077 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.810078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.810085 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.811950 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.812360 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.812364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.812364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.812423 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.812619 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.812620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.812620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.812785 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.812934 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.813492 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.813567 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":174,"thread_start_us":143,"threads_started":1}
I20260812 04:02:07.813683 25392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.252@KRBTEST.COM for kudu/127.31.43.253@KRBTEST.COM
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.254@KRBTEST.COM
I20260812 04:02:07.821425 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.814093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52278 (local address 127.31.43.253:46791)
0812 04:02:07.814138 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.814140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.814151 (+ 11us) server_negotiation.cc:407] Connection header received
0812 04:02:07.814300 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.814301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.814314 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.814380 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.814665 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.815413 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.816268 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.816612 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.816684 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.818814 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.818816 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.818816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.818825 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.820230 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.820626 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.820628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.820628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.820694 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.820935 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.820936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.820936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.821054 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.821109 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.821295 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.821357 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 04:02:07.821477 25394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.814028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.253:46791 (local address 127.0.0.1:52278)
0812 04:02:07.814084 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.814097 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:07.814141 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:07.814380 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:07.814381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:07.814519 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:07.814568 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.814570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.815492 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.815493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.816193 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.816195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.816691 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.816692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.816740 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.817151 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:07.817156 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:07.818736 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:07.820306 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.820307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.820307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.820536 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.820771 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.820772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.820773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.820861 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.821126 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:07.821127 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:07.821177 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 04:02:07.821260 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 04:02:07.821358 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 04:02:07.821914 25392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } has no permanent_uuid. Determining permanent_uuid...
I20260812 04:02:07.823060 25226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.814472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49928 (local address 127.31.43.254:35595)
0812 04:02:07.814548 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.814550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.814558 (+ 8us) server_negotiation.cc:407] Connection header received
0812 04:02:07.814579 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.814580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.814590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.814648 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.815270 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.815928 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.816893 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.817136 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.817207 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.819816 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.819817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.819817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.819824 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.821820 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.822189 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.822190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.822191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.822268 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.822470 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.822471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.822471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.822627 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.822830 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.822866 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.822958 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.252@KRBTEST.COM for kudu/127.31.43.252@KRBTEST.COM
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.43.253@KRBTEST.COM
I20260812 04:02:07.829239 25394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.822187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46816)
0812 04:02:07.822241 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.822253 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:07.822298 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:07.822410 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:07.822411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:07.822570 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:07.822607 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.822609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.823663 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.823664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.824153 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.824155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.824581 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.824582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.824637 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.825008 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:07.825012 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:07.826357 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:07.828003 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.828004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.828005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.828187 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.828624 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.828625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.828625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.828676 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.828990 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:07.828991 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:07.829039 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 04:02:07.829096 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 04:02:07.829170 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 04:02:07.829360 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.822265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.31.43.252:34071)
0812 04:02:07.822303 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.822305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.822323 (+ 18us) server_negotiation.cc:407] Connection header received
0812 04:02:07.822344 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.822345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.822355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.822404 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.822718 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.823582 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.824238 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.824504 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.824560 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.826451 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.826452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.826452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.826459 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.827922 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.828477 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.828478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.828479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.828550 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.828742 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.828743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.828744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.828865 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.828984 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.829148 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.829229 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 04:02:07.830842 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.823704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52290 (local address 127.31.43.253:46791)
0812 04:02:07.823743 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.823745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.823752 (+ 7us) server_negotiation.cc:407] Connection header received
0812 04:02:07.823770 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.823771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.823779 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.823826 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.824313 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.824873 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.825586 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.825775 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.825877 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.828031 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.828039 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.828039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.828046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.829659 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.830048 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.830050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.830050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.830106 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.830315 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.830316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.830316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.830432 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.830605 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.830644 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.830741 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 04:02:07.831067 25392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e: Bootstrap starting.
I20260812 04:02:07.832125 25392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e: Neither blocks nor log segments found. Creating new log.
I20260812 04:02:07.832373 31919 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 04:02:07.832448 25392 log.cc:824] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e: Log is configured to *not* fsync() on all Append() calls
I20260812 04:02:07.833341 25392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e: No bootstrap required, opened a new log
I20260812 04:02:07.833489 25193 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5b357bca90bd495b93426519842238ef"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49928
I20260812 04:02:07.835582 25392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } }
I20260812 04:02:07.835747 25392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99032a6e18374f6b88c29dbb15cb838e, State: Initialized, Role: FOLLOWER
I20260812 04:02:07.835829 25392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } }
I20260812 04:02:07.836186 25392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:02:07.836155 25400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } } }
I20260812 04:02:07.836280 25400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [sys.catalog]: This master's current role is: FOLLOWER
W20260812 04:02:07.838547 25412 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:02:07.838621 25412 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:02:07.838653 25315 master_runner.cc:186] Error getting master registration for 127.31.43.254:35595: OK, instance_id { permanent_uuid: "5b357bca90bd495b93426519842238ef" instance_seqno: 1786507327449731 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 04:02:07.840987 25315 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 04:02:07.861133 25308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } attempt: 3
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.253@KRBTEST.COM for kudu/127.31.43.252@KRBTEST.COM
I20260812 04:02:07.870018 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.861637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46826 (local address 127.31.43.252:34071)
0812 04:02:07.861690 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.861692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.861704 (+ 12us) server_negotiation.cc:407] Connection header received
0812 04:02:07.861783 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.861784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.861793 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.861843 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.862287 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.863079 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.864103 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.864406 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.864472 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.866929 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.866931 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.866931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.866939 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.868663 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.869008 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.869010 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.869010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.869127 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.869387 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.869388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.869388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.869606 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.869674 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.869867 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.869946 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 04:02:07.870092 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.861548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46826)
0812 04:02:07.861628 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.861645 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:07.861717 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:07.861877 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:07.861878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:07.862104 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:07.862190 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.862193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.863168 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.863169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.864005 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.864008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.864483 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.864484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.864548 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.865087 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:07.865093 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:07.866831 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:07.868741 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.868743 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.868743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.868927 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.869218 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.869219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.869219 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.869304 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.869681 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:07.869682 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:07.869758 (+ 76us) client_negotiation.cc:769] Sending connection context
0812 04:02:07.869841 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 04:02:07.869975 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 04:02:07.872196 25308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192: Bootstrap starting.
I20260812 04:02:07.873003 25308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192: Neither blocks nor log segments found. Creating new log.
I20260812 04:02:07.873319 25308 log.cc:824] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192: Log is configured to *not* fsync() on all Append() calls
I20260812 04:02:07.874138 25308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192: No bootstrap required, opened a new log
I20260812 04:02:07.875783 25308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } }
I20260812 04:02:07.875907 25308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 885cd073447a4577bd2b21c513e86192, State: Initialized, Role: FOLLOWER
I20260812 04:02:07.875998 25308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } }
I20260812 04:02:07.876318 25415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } } }
I20260812 04:02:07.876398 25308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:02:07.876415 25415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 04:02:07.878255 25426 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:02:07.878319 25426 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:02:07.878587 25230 master_runner.cc:186] Error getting master registration for 127.31.43.254:35595: OK, instance_id { permanent_uuid: "5b357bca90bd495b93426519842238ef" instance_seqno: 1786507327449731 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 04:02:07.881536 25230 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 04:02:07.895606 25221 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } attempt: 3
Aug 12 04:02:07 dist-test-slave-8tdl krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507327, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.43.254@KRBTEST.COM for kudu/127.31.43.252@KRBTEST.COM
I20260812 04:02:07.903560 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.895987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.43.252:34071 (local address 127.0.0.1:46836)
0812 04:02:07.896061 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 04:02:07.896075 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 04:02:07.896123 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 04:02:07.896271 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 04:02:07.896272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 04:02:07.896476 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 04:02:07.896533 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.896535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.897474 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.897475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.897974 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 04:02:07.897975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.898420 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.898421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 04:02:07.898472 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.898951 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 04:02:07.898956 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 04:02:07.900309 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 04:02:07.902410 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.902411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.902411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.902599 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.902844 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 04:02:07.902845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 04:02:07.902845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 04:02:07.902928 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 04:02:07.903204 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 04:02:07.903205 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 04:02:07.903253 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 04:02:07.903338 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 04:02:07.903449 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 04:02:07.903553 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.896076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46836 (local address 127.31.43.252:34071)
0812 04:02:07.896133 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.896135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.896148 (+ 13us) server_negotiation.cc:407] Connection header received
0812 04:02:07.896192 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.896193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.896203 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.896253 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.896608 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.897396 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.898069 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.898360 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.898422 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.900412 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.900414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.900414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.900422 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.902320 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.902703 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.902704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.902705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.902764 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.902998 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.902999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.903000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.903147 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.903211 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.903381 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.903454 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 04:02:07.905761 25221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef: Bootstrap starting.
I20260812 04:02:07.906576 25221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef: Neither blocks nor log segments found. Creating new log.
I20260812 04:02:07.906942 25221 log.cc:824] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef: Log is configured to *not* fsync() on all Append() calls
I20260812 04:02:07.907859 25221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef: No bootstrap required, opened a new log
I20260812 04:02:07.909312 25221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } }
I20260812 04:02:07.909471 25221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b357bca90bd495b93426519842238ef, State: Initialized, Role: FOLLOWER
I20260812 04:02:07.909574 25221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } }
I20260812 04:02:07.909932 25221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [sys.catalog]: configured and running, proceeding with master startup.
I20260812 04:02:07.909962 25193 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 04:02:07.910022 25193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:02:07.911044 25193 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } }
I20260812 04:02:07.911342 25193 leader_election.cc:288] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [CANDIDATE]: Term 1 election: Requested vote from peers 885cd073447a4577bd2b21c513e86192 (127.31.43.253:46791), 99032a6e18374f6b88c29dbb15cb838e (127.31.43.252:34071)
W20260812 04:02:07.911974 25443 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 04:02:07.912026 25443 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 04:02:07.912042 25278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b357bca90bd495b93426519842238ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "885cd073447a4577bd2b21c513e86192"
I20260812 04:02:07.912226 25278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:02:07.913164 25278 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b357bca90bd495b93426519842238ef in term 1.
I20260812 04:02:07.913429 25361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b357bca90bd495b93426519842238ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "99032a6e18374f6b88c29dbb15cb838e"
I20260812 04:02:07.913556 25361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [term 0 FOLLOWER]: Advancing to term 1
I20260812 04:02:07.913563 25166 leader_election.cc:302] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b357bca90bd495b93426519842238ef, 885cd073447a4577bd2b21c513e86192; no voters:
I20260812 04:02:07.914476 25361 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b357bca90bd495b93426519842238ef in term 1.
I20260812 04:02:07.915499 25147 master_runner.cc:263] No leader master found from master 5b357bca90bd495b93426519842238ef
I20260812 04:02:07.915551 25147 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 04:02:07.918875 25429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } } }
I20260812 04:02:07.919117 25429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:02:07.919227 25429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [term 1 FOLLOWER]: Leader election won for term 1
I20260812 04:02:07.919324 25429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [term 1 LEADER]: Becoming Leader. State: Replica: 5b357bca90bd495b93426519842238ef, State: Running, Role: LEADER
I20260812 04:02:07.919461 25429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } }
I20260812 04:02:07.920001 25429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b357bca90bd495b93426519842238ef. Latest consensus state: current_term: 1 leader_uuid: "5b357bca90bd495b93426519842238ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } } }
I20260812 04:02:07.920149 25429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [sys.catalog]: This master's current role is: LEADER
I20260812 04:02:07.930800 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.918957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.31.43.253:46791)
0812 04:02:07.919004 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.919006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.922894 (+ 3888us) server_negotiation.cc:407] Connection header received
0812 04:02:07.923007 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.923008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.923029 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.923061 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.923898 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.924669 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.925757 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.926071 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.926112 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.927818 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.927819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.927820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.927827 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.929506 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.929904 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.929905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.929906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.929961 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.930173 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.930174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.930175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.930304 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.930376 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.930577 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.930661 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 04:02:07.936483 25226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.920020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49936 (local address 127.31.43.254:35595)
0812 04:02:07.920061 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.920064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.924421 (+ 4357us) server_negotiation.cc:407] Connection header received
0812 04:02:07.924521 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.924522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.924534 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.924563 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.927075 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.927743 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.931489 (+ 3746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.931808 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.931853 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.933518 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.933519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.933520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.933527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.935246 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.935634 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.935635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.935635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.935690 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.935878 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.935878 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.935879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.936011 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.936106 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.936302 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.936383 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 04:02:07.937770 25456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 04:02:07.938107 25456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260812 04:02:07.947453 25170 tablet.cc:2369] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 04:02:07.948062 25361 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [term 1 FOLLOWER]: Refusing update from remote peer 5b357bca90bd495b93426519842238ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:02:07.948449 25429 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:02:07.949123 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:07.919324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.31.43.252:34071)
0812 04:02:07.919368 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:07.919371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:07.923414 (+ 4043us) server_negotiation.cc:407] Connection header received
0812 04:02:07.923525 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:07.923526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:07.923538 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:07.923569 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:07.925336 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.926021 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.931947 (+ 5926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:07.932250 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:07.932326 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:07.935897 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 04:02:07.935899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 04:02:07.935900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 04:02:07.935908 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 04:02:07.938194 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.947879 (+ 9685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.947881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.947881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.947988 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 04:02:07.948362 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 04:02:07.948364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 04:02:07.948364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 04:02:07.948519 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 04:02:07.948611 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:07.948912 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 04:02:07.949026 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 04:02:07.949949 25400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b357bca90bd495b93426519842238ef. Latest consensus state: current_term: 1 leader_uuid: "5b357bca90bd495b93426519842238ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } } }
I20260812 04:02:07.950039 25400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:02:07.950484 25278 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [term 1 FOLLOWER]: Refusing update from remote peer 5b357bca90bd495b93426519842238ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 04:02:07.950802 25429 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 04:02:07.952172 25415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b357bca90bd495b93426519842238ef. Latest consensus state: current_term: 1 leader_uuid: "5b357bca90bd495b93426519842238ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } } }
I20260812 04:02:07.952276 25415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:02:07.953869 25450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b357bca90bd495b93426519842238ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } } }
I20260812 04:02:07.954017 25450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [sys.catalog]: This master's current role is: LEADER
I20260812 04:02:07.954464 25400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b357bca90bd495b93426519842238ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } } }
I20260812 04:02:07.954564 25400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:02:07.954883 25457 mvcc.cc:205] Tried to move back new op lower bound from 7317534015271608320 to 7317534015159259136. Current Snapshot: MvccSnapshot[applied={T|T < 7317534015271608320}]
I20260812 04:02:07.955031 25458 mvcc.cc:205] Tried to move back new op lower bound from 7317534015271608320 to 7317534015159259136. Current Snapshot: MvccSnapshot[applied={T|T < 7317534015271608320}]
I20260812 04:02:07.956597 25429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b357bca90bd495b93426519842238ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } } }
I20260812 04:02:07.956696 25429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef [sys.catalog]: This master's current role is: LEADER
I20260812 04:02:07.957173 25415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b357bca90bd495b93426519842238ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b357bca90bd495b93426519842238ef" member_type: VOTER last_known_addr { host: "127.31.43.254" port: 35595 } } peers { permanent_uuid: "885cd073447a4577bd2b21c513e86192" member_type: VOTER last_known_addr { host: "127.31.43.253" port: 46791 } } peers { permanent_uuid: "99032a6e18374f6b88c29dbb15cb838e" member_type: VOTER last_known_addr { host: "127.31.43.252" port: 34071 } } }
I20260812 04:02:07.957269 25415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 04:02:07.957607 25456 catalog_manager.cc:1347] Generated new cluster ID: a42a15fe7ee444e28d6ecc5248b91dac
I20260812 04:02:07.957657 25456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 04:02:07.957505 25461 mvcc.cc:205] Tried to move back new op lower bound from 7317534015271608320 to 7317534015159259136. Current Snapshot: MvccSnapshot[applied={T|T < 7317534015271608320}]
I20260812 04:02:07.978343 25456 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 04:02:07.979204 25456 catalog_manager.cc:1504] Loading token signing keys...
I20260812 04:02:07.989565 25456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b357bca90bd495b93426519842238ef: Generated new TSK 0
I20260812 04:02:07.989810 25456 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 04:02:08.021349 25226 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.019639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49948 (local address 127.31.43.254:35595)
0812 04:02:08.019687 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.019689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.020618 (+ 929us) server_negotiation.cc:407] Connection header received
0812 04:02:08.020723 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.020724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.020736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.020766 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.021267 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49948: BlockingRecv error: recv got EOF from 127.0.0.1:49948 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 04:02:08.022100 25312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.020062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.31.43.253:46791)
0812 04:02:08.020102 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.020104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.021609 (+ 1505us) server_negotiation.cc:407] Connection header received
0812 04:02:08.021703 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.021704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.021715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.021744 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.022039 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52300: BlockingRecv error: recv got EOF from 127.0.0.1:52300 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 04:02:08.022940 25398 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.020355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.31.43.252:34071)
0812 04:02:08.020394 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.020396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.022304 (+ 1908us) server_negotiation.cc:407] Connection header received
0812 04:02:08.022401 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.022403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.022418 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.022457 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.022855 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46862: BlockingRecv error: recv got EOF from 127.0.0.1:46862 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 04:02:08.025724 25312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.024914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52308 (local address 127.31.43.253:46791)
0812 04:02:08.024955 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.024957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.025229 (+ 272us) server_negotiation.cc:407] Connection header received
0812 04:02:08.025330 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.025331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.025351 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.025379 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.025678 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52308: BlockingRecv error: recv got EOF from 127.0.0.1:52308 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 04:02:08.028524 25312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.028041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52314 (local address 127.31.43.253:46791)
0812 04:02:08.028079 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.028080 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.028089 (+ 9us) server_negotiation.cc:407] Connection header received
0812 04:02:08.028145 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.028146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.028156 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.028205 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.028473 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52314: BlockingRecv error: recv got EOF from 127.0.0.1:52314 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 04:02:08.032262 25312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.031661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52316 (local address 127.31.43.253:46791)
0812 04:02:08.031719 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.031721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.031760 (+ 39us) server_negotiation.cc:407] Connection header received
0812 04:02:08.031840 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.031840 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.031850 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.031903 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.032199 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52316: BlockingRecv error: recv got EOF from 127.0.0.1:52316 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 04:02:08.038203 25312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.037446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52320 (local address 127.31.43.253:46791)
0812 04:02:08.037492 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.037494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.037631 (+ 137us) server_negotiation.cc:407] Connection header received
0812 04:02:08.037717 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.037718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.037730 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.037797 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.038146 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52320: BlockingRecv error: recv got EOF from 127.0.0.1:52320 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 04:02:08.048615 25312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.047808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52334 (local address 127.31.43.253:46791)
0812 04:02:08.047861 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.047873 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.048016 (+ 143us) server_negotiation.cc:407] Connection header received
0812 04:02:08.048093 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.048094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.048106 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.048174 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.048536 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52334: BlockingRecv error: recv got EOF from 127.0.0.1:52334 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 04:02:08.066606 25312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.065820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52350 (local address 127.31.43.253:46791)
0812 04:02:08.065871 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.065873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.066016 (+ 143us) server_negotiation.cc:407] Connection header received
0812 04:02:08.066096 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.066097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.066110 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.066181 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.066533 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52350: BlockingRecv error: recv got EOF from 127.0.0.1:52350 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 04:02:08.100627 25312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.099818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52360 (local address 127.31.43.253:46791)
0812 04:02:08.099873 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.099876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.100021 (+ 145us) server_negotiation.cc:407] Connection header received
0812 04:02:08.100106 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.100107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.100123 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.100189 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.100552 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52360: BlockingRecv error: recv got EOF from 127.0.0.1:52360 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 04:02:08.167809 25312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.166074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52372 (local address 127.31.43.253:46791)
0812 04:02:08.166149 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.166151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.166285 (+ 134us) server_negotiation.cc:407] Connection header received
0812 04:02:08.166329 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.166330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.166343 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.166419 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.167759 (+ 1340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52372: BlockingRecv error: recv got EOF from 127.0.0.1:52372 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 04:02:08.297030 25312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.296110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52378 (local address 127.31.43.253:46791)
0812 04:02:08.296186 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.296188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.296287 (+ 99us) server_negotiation.cc:407] Connection header received
0812 04:02:08.296357 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.296358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.296370 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.296449 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.296864 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52378: BlockingRecv error: recv got EOF from 127.0.0.1:52378 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 04:02:08.555128 25312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 04:02:08.554326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52390 (local address 127.31.43.253:46791)
0812 04:02:08.554404 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:08.554406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:08.554417 (+ 11us) server_negotiation.cc:407] Connection header received
0812 04:02:08.554556 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:08.554558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:08.554569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:08.554636 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 04:02:08.555039 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52390: BlockingRecv error: recv got EOF from 127.0.0.1:52390 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 04:02:08.839164 25412 catalog_manager.cc:1259] Loaded cluster ID: a42a15fe7ee444e28d6ecc5248b91dac
I20260812 04:02:08.839255 25412 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e: loading cluster ID for follower catalog manager: success
I20260812 04:02:08.840062 25412 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e: acquiring CA information for follower catalog manager: success
I20260812 04:02:08.840341 25412 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 99032a6e18374f6b88c29dbb15cb838e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 04:02:08.878885 25426 catalog_manager.cc:1259] Loaded cluster ID: a42a15fe7ee444e28d6ecc5248b91dac
I20260812 04:02:08.879053 25426 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192: loading cluster ID for follower catalog manager: success
I20260812 04:02:08.879956 25426 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192: acquiring CA information for follower catalog manager: success
I20260812 04:02:08.880241 25426 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 885cd073447a4577bd2b21c513e86192: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 04:02:09.071345 25538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:09.068429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52392 (local address 127.31.43.253:46791)
0812 04:02:09.068550 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:09.068551 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:09.068617 (+ 66us) server_negotiation.cc:407] Connection header received
0812 04:02:09.068663 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:09.068664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:09.068728 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:09.068805 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:02:09.068971 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:09.069478 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:09.070261 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:09.070465 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:09.070522 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:09.070797 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:02:09.070968 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:02:09.071037 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:09.071158 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 04:02:09.071182 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260812 04:02:09.075770 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:09.073352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46876 (local address 127.31.43.252:34071)
0812 04:02:09.073472 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:09.073474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:09.073483 (+ 9us) server_negotiation.cc:407] Connection header received
0812 04:02:09.073541 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:09.073541 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:09.073579 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:09.073650 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:02:09.073738 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:09.074207 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:09.075023 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:09.075233 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:09.075294 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:09.075449 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:02:09.075558 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:02:09.075616 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:09.075651 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 04:02:09.075669 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
I20260812 04:02:09.080507 25538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:09.077651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52396 (local address 127.31.43.253:46791)
0812 04:02:09.077701 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:09.077704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:09.077823 (+ 119us) server_negotiation.cc:407] Connection header received
0812 04:02:09.077934 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:09.077935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:09.077950 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:09.077981 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:02:09.078165 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:09.078963 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:09.079772 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:09.080000 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:09.080079 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:09.080217 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:02:09.080251 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:02:09.080300 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:09.080399 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 04:02:09.080425 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 04:02:09.081157 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:09.077915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46888 (local address 127.31.43.252:34071)
0812 04:02:09.077950 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:09.077952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:09.078190 (+ 238us) server_negotiation.cc:407] Connection header received
0812 04:02:09.078295 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:09.078296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:09.078308 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:09.078362 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:02:09.078491 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:09.079193 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:09.080278 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:09.080540 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:09.080617 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:09.080843 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:02:09.080880 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:02:09.080950 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:09.081050 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 04:02:09.081079 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 04:02:09.081612 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 04:02:09.077667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49950 (local address 127.31.43.254:35595)
0812 04:02:09.077816 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 04:02:09.077819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 04:02:09.077883 (+ 64us) server_negotiation.cc:407] Connection header received
0812 04:02:09.077977 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 04:02:09.077978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 04:02:09.078036 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 04:02:09.078316 (+ 280us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 04:02:09.078361 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:09.079065 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:09.080264 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 04:02:09.080588 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 04:02:09.080913 (+ 325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 04:02:09.081134 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 04:02:09.081233 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 04:02:09.081317 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 04:02:09.081485 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 04:02:09.081532 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260812 04:02:09.083382 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 25147
I20260812 04:02:09.092696 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 25230
I20260812 04:02:09.101339 31919 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaEzvCg/build/release/bin/kudu with pid 25315
2026-08-12T04:02:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3068 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6022 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (287246 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST